OLMo 1B hf

Allen AI · allen-ai/olmo-1b-hf

ProviderAllen AI
Familyolmo
Typellm-chat
Statusactive
Released2024-04-12

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed34.6
BIG-Bench Hardunassessed30.5
GPQA Diamondunassessed26.2
GSM8Kunassessed1.9
HellaSwagunassessed63.6
IFEvalunassessed21.8
MATH-500unassessed1.7
MMLU: Abstract Algebraunassessed29.0
MMLU: Anatomyunassessed32.6
MMLU: Astronomyunassessed20.4
MMLU: Business Ethicsunassessed23.0
MMLU: Clinical Knowledgeunassessed19.6
MMLU: College Biologyunassessed25.7
MMLU: College Chemistryunassessed20.0
MMLU: College Computer Scienceunassessed36.0
MMLU: College Mathematicsunassessed26.0
MMLU: College Medicineunassessed27.2
MMLU: College Physicsunassessed22.5
MMLU: Computer Securityunassessed27.0
MMLU: Conceptual Physicsunassessed23.0
MMLU: Econometricsunassessed23.7
MMLU: Electrical Engineeringunassessed30.3
MMLU: Elementary Mathematicsunassessed24.6
MMLU: Formal Logicunassessed20.6
MMLU: Global Factsunassessed34.0
MMLU: High School Biologyunassessed29.7
MMLU: High School Chemistryunassessed28.6
MMLU: High School Computer Scienceunassessed31.0
MMLU: High School European Historyunassessed24.8
MMLU: High School Geographyunassessed18.2
MMLU: High School Government and Politicsunassessed25.9
MMLU: High School Macroeconomicsunassessed27.2
MMLU: High School Mathematicsunassessed25.9
MMLU: High School Microeconomicsunassessed21.4
MMLU: High School Physicsunassessed31.1
MMLU: High School Psychologyunassessed24.0
MMLU: High School Statisticsunassessed36.6
MMLU: High School US Historyunassessed27.9
MMLU: High School World Historyunassessed28.3
MMLU: Human Agingunassessed21.5
MMLU: Human Sexualityunassessed21.4
MMLU: International Lawunassessed28.1
MMLU: Jurisprudenceunassessed24.1
MMLU: Logical Fallaciesunassessed27.0
MMLU: Machine Learningunassessed33.0
MMLU: Managementunassessed22.3
MMLU: Marketingunassessed25.2
MMLU: Medical Geneticsunassessed27.0
MMLU: Miscellaneousunassessed29.0
MMLU: Moral Disputesunassessed25.1
MMLU: Moral Scenariosunassessed24.7
MMLU: Nutritionunassessed29.7
MMLU: Philosophyunassessed28.9
MMLU: Prehistoryunassessed25.6
MMLU-Prounassessed11.7
MMLU: Professional Accountingunassessed22.0
MMLU: Professional Lawunassessed24.1
MMLU: Professional Medicineunassessed40.4
MMLU: Professional Psychologyunassessed27.5
MMLU: Public Relationsunassessed24.5
MMLU: Security Studiesunassessed21.2
MMLU: Sociologyunassessed22.4
MMLU: US Foreign Policyunassessed30.0
MMLU: Virologyunassessed22.9
MMLU: World Religionsunassessed25.7
MuSRunassessed41.0
TruthfulQAunassessed32.9
WinoGrandeunassessed61.1

Data

This card as JSON · Edit on GitHub