gemma 7B it

Google DeepMind · google/gemma-7b-it

ProviderGoogle DeepMind
Familygemma
Typellm-chat
Statusactive
Released2024-02-13

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed61.1
BIG-Bench Hardunassessed36.5
GPQA Diamondunassessed28.4
GSM8Kunassessed52.8
HellaSwagunassessed82.5
IFEvalunassessed38.7
MATH-500unassessed2.9
MMLU: Abstract Algebraunassessed31.0
MMLU: Anatomyunassessed57.8
MMLU: Astronomyunassessed73.0
MMLU: Business Ethicsunassessed64.0
MMLU: Clinical Knowledgeunassessed68.7
MMLU: College Biologyunassessed77.1
MMLU: College Chemistryunassessed47.0
MMLU: College Computer Scienceunassessed55.0
MMLU: College Mathematicsunassessed36.0
MMLU: College Medicineunassessed69.9
MMLU: College Physicsunassessed40.2
MMLU: Computer Securityunassessed73.0
MMLU: Conceptual Physicsunassessed62.1
MMLU: Econometricsunassessed48.2
MMLU: Electrical Engineeringunassessed62.8
MMLU: Elementary Mathematicsunassessed50.3
MMLU: Formal Logicunassessed50.0
MMLU: Global Factsunassessed43.0
MMLU: High School Biologyunassessed80.3
MMLU: High School Chemistryunassessed52.2
MMLU: High School Computer Scienceunassessed67.0
MMLU: High School European Historyunassessed76.4
MMLU: High School Geographyunassessed82.3
MMLU: High School Government and Politicsunassessed88.6
MMLU: High School Macroeconomicsunassessed64.6
MMLU: High School Mathematicsunassessed43.0
MMLU: High School Microeconomicsunassessed66.8
MMLU: High School Physicsunassessed41.7
MMLU: High School Psychologyunassessed83.1
MMLU: High School Statisticsunassessed57.9
MMLU: High School US Historyunassessed83.3
MMLU: High School World Historyunassessed85.2
MMLU: Human Agingunassessed71.7
MMLU: Human Sexualityunassessed72.5
MMLU: International Lawunassessed84.3
MMLU: Jurisprudenceunassessed77.8
MMLU: Logical Fallaciesunassessed74.8
MMLU: Machine Learningunassessed51.8
MMLU: Managementunassessed84.5
MMLU: Marketingunassessed89.7
MMLU: Medical Geneticsunassessed72.0
MMLU: Miscellaneousunassessed83.8
MMLU: Moral Disputesunassessed71.7
MMLU: Moral Scenariosunassessed40.3
MMLU: Nutritionunassessed76.8
MMLU: Philosophyunassessed72.3
MMLU: Prehistoryunassessed73.8
MMLU-Prounassessed16.9
MMLU: Professional Accountingunassessed49.3
MMLU: Professional Lawunassessed48.1
MMLU: Professional Medicineunassessed63.2
MMLU: Professional Psychologyunassessed68.8
MMLU: Public Relationsunassessed70.9
MMLU: Security Studiesunassessed74.7
MMLU: Sociologyunassessed85.6
MMLU: US Foreign Policyunassessed88.0
MMLU: Virologyunassessed53.6
MMLU: World Religionsunassessed81.9
MuSRunassessed42.7
TruthfulQAunassessed44.9
WinoGrandeunassessed78.5

Data

This card as JSON · Edit on GitHub