gemma 2B it

Google DeepMind · google/gemma-2b-it

ProviderGoogle DeepMind
Familygemma
Typellm-chat
Statusactive
Released2024-02-08

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed43.9
BIG-Bench Hardunassessed31.5
GPQA Diamondunassessed27.9
GSM8Kunassessed5.5
HellaSwagunassessed62.7
IFEvalunassessed26.9
MATH-500unassessed2.0
MMLU: Abstract Algebraunassessed28.0
MMLU: Anatomyunassessed38.5
MMLU: Astronomyunassessed33.6
MMLU: Business Ethicsunassessed48.0
MMLU: Clinical Knowledgeunassessed42.3
MMLU: College Biologyunassessed34.0
MMLU: College Chemistryunassessed26.0
MMLU: College Computer Scienceunassessed31.0
MMLU: College Mathematicsunassessed26.0
MMLU: College Medicineunassessed35.8
MMLU: College Physicsunassessed18.6
MMLU: Computer Securityunassessed45.0
MMLU: Conceptual Physicsunassessed35.3
MMLU: Econometricsunassessed28.1
MMLU: Electrical Engineeringunassessed46.2
MMLU: Elementary Mathematicsunassessed24.9
MMLU: Formal Logicunassessed25.4
MMLU: Global Factsunassessed30.0
MMLU: High School Biologyunassessed31.6
MMLU: High School Chemistryunassessed29.1
MMLU: High School Computer Scienceunassessed37.0
MMLU: High School European Historyunassessed46.1
MMLU: High School Geographyunassessed46.0
MMLU: High School Government and Politicsunassessed47.7
MMLU: High School Macroeconomicsunassessed32.8
MMLU: High School Mathematicsunassessed20.0
MMLU: High School Microeconomicsunassessed34.0
MMLU: High School Physicsunassessed25.8
MMLU: High School Psychologyunassessed51.0
MMLU: High School Statisticsunassessed20.4
MMLU: High School US Historyunassessed42.6
MMLU: High School World Historyunassessed51.9
MMLU: Human Agingunassessed39.0
MMLU: Human Sexualityunassessed42.7
MMLU: International Lawunassessed52.1
MMLU: Jurisprudenceunassessed46.3
MMLU: Logical Fallaciesunassessed36.2
MMLU: Machine Learningunassessed33.0
MMLU: Managementunassessed44.7
MMLU: Marketingunassessed59.4
MMLU: Medical Geneticsunassessed39.0
MMLU: Miscellaneousunassessed46.2
MMLU: Moral Disputesunassessed40.8
MMLU: Moral Scenariosunassessed25.3
MMLU: Nutritionunassessed45.1
MMLU: Philosophyunassessed40.2
MMLU: Prehistoryunassessed40.7
MMLU-Prounassessed13.5
MMLU: Professional Accountingunassessed30.5
MMLU: Professional Lawunassessed31.9
MMLU: Professional Medicineunassessed20.2
MMLU: Professional Psychologyunassessed38.2
MMLU: Public Relationsunassessed41.8
MMLU: Security Studiesunassessed47.3
MMLU: Sociologyunassessed43.8
MMLU: US Foreign Policyunassessed64.0
MMLU: Virologyunassessed41.0
MMLU: World Religionsunassessed43.9
MuSRunassessed33.4
TruthfulQAunassessed45.8
WinoGrandeunassessed60.9

Data

This card as JSON · Edit on GitHub