gemma 2B

Google DeepMind · google/gemma-2b

ProviderGoogle DeepMind
Familygemma
Typellm-chat
Statusactive
Released2024-02-08

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed48.4
BIG-Bench Hardunassessed33.7
GPQA Diamondunassessed25.5
GSM8Kunassessed16.9
HellaSwagunassessed71.8
IFEvalunassessed20.4
MATH-500unassessed3.0
MMLU: Abstract Algebraunassessed26.0
MMLU: Anatomyunassessed48.9
MMLU: Astronomyunassessed42.8
MMLU: Business Ethicsunassessed44.0
MMLU: Clinical Knowledgeunassessed46.8
MMLU: College Biologyunassessed45.8
MMLU: College Chemistryunassessed37.0
MMLU: College Computer Scienceunassessed38.0
MMLU: College Mathematicsunassessed30.0
MMLU: College Medicineunassessed42.8
MMLU: College Physicsunassessed14.7
MMLU: Computer Securityunassessed53.0
MMLU: Conceptual Physicsunassessed41.7
MMLU: Econometricsunassessed31.6
MMLU: Electrical Engineeringunassessed40.7
MMLU: Elementary Mathematicsunassessed26.2
MMLU: Formal Logicunassessed27.0
MMLU: Global Factsunassessed31.0
MMLU: High School Biologyunassessed48.4
MMLU: High School Chemistryunassessed40.9
MMLU: High School Computer Scienceunassessed42.0
MMLU: High School European Historyunassessed41.8
MMLU: High School Geographyunassessed50.5
MMLU: High School Government and Politicsunassessed59.1
MMLU: High School Macroeconomicsunassessed41.0
MMLU: High School Mathematicsunassessed25.9
MMLU: High School Microeconomicsunassessed38.7
MMLU: High School Physicsunassessed25.2
MMLU: High School Psychologyunassessed57.2
MMLU: High School Statisticsunassessed35.2
MMLU: High School US Historyunassessed44.1
MMLU: High School World Historyunassessed39.7
MMLU: Human Agingunassessed44.4
MMLU: Human Sexualityunassessed45.8
MMLU: International Lawunassessed61.2
MMLU: Jurisprudenceunassessed41.7
MMLU: Logical Fallaciesunassessed41.1
MMLU: Machine Learningunassessed39.3
MMLU: Managementunassessed56.3
MMLU: Marketingunassessed60.7
MMLU: Medical Geneticsunassessed43.0
MMLU: Miscellaneousunassessed54.7
MMLU: Moral Disputesunassessed43.9
MMLU: Moral Scenariosunassessed23.7
MMLU: Nutritionunassessed46.7
MMLU: Philosophyunassessed41.2
MMLU: Prehistoryunassessed46.3
MMLU-Prounassessed13.7
MMLU: Professional Accountingunassessed34.0
MMLU: Professional Lawunassessed34.7
MMLU: Professional Medicineunassessed35.3
MMLU: Professional Psychologyunassessed37.4
MMLU: Public Relationsunassessed47.3
MMLU: Security Studiesunassessed46.1
MMLU: Sociologyunassessed42.8
MMLU: US Foreign Policyunassessed57.0
MMLU: Virologyunassessed44.6
MMLU: World Religionsunassessed54.4
MuSRunassessed39.8
TruthfulQAunassessed33.1
WinoGrandeunassessed66.3

Data

This card as JSON · Edit on GitHub