Mistral 7B Instruct v0.2

Mistral AI · mistral/mistral-7b-instruct-v0-2

ProviderMistral AI
Familymistral
Typellm-chat
Statusactive
Released2023-12-11

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed63.1
BIG-Bench Hardunassessed44.6
GPQA Diamondunassessed27.6
GSM8Kunassessed40.0
HellaSwagunassessed84.9
IFEvalunassessed55.0
MATH-500unassessed3.0
MMLU: Abstract Algebraunassessed32.0
MMLU: Anatomyunassessed57.0
MMLU: Astronomyunassessed62.5
MMLU: Business Ethicsunassessed60.0
MMLU: Clinical Knowledgeunassessed67.2
MMLU: College Biologyunassessed69.4
MMLU: College Chemistryunassessed40.0
MMLU: College Computer Scienceunassessed53.0
MMLU: College Mathematicsunassessed39.0
MMLU: College Medicineunassessed59.0
MMLU: College Physicsunassessed42.2
MMLU: Computer Securityunassessed69.0
MMLU: Conceptual Physicsunassessed53.2
MMLU: Econometricsunassessed40.4
MMLU: Electrical Engineeringunassessed61.4
MMLU: Elementary Mathematicsunassessed36.8
MMLU: Formal Logicunassessed42.1
MMLU: Global Factsunassessed35.0
MMLU: High School Biologyunassessed63.2
MMLU: High School Chemistryunassessed51.2
MMLU: High School Computer Scienceunassessed65.0
MMLU: High School European Historyunassessed73.9
MMLU: High School Geographyunassessed76.3
MMLU: High School Government and Politicsunassessed85.5
MMLU: High School Macroeconomicsunassessed55.6
MMLU: High School Mathematicsunassessed30.0
MMLU: High School Microeconomicsunassessed65.5
MMLU: High School Physicsunassessed35.8
MMLU: High School Psychologyunassessed79.1
MMLU: High School Statisticsunassessed44.4
MMLU: High School US Historyunassessed76.5
MMLU: High School World Historyunassessed75.5
MMLU: Human Agingunassessed61.9
MMLU: Human Sexualityunassessed74.0
MMLU: International Lawunassessed81.0
MMLU: Jurisprudenceunassessed74.1
MMLU: Logical Fallaciesunassessed73.0
MMLU: Machine Learningunassessed44.6
MMLU: Managementunassessed75.7
MMLU: Marketingunassessed86.3
MMLU: Medical Geneticsunassessed67.0
MMLU: Miscellaneousunassessed78.0
MMLU: Moral Disputesunassessed69.7
MMLU: Moral Scenariosunassessed31.2
MMLU: Nutritionunassessed68.6
MMLU: Philosophyunassessed70.4
MMLU: Prehistoryunassessed70.1
MMLU-Prounassessed27.2
MMLU: Professional Accountingunassessed45.0
MMLU: Professional Lawunassessed43.5
MMLU: Professional Medicineunassessed61.8
MMLU: Professional Psychologyunassessed63.1
MMLU: Public Relationsunassessed70.9
MMLU: Security Studiesunassessed70.6
MMLU: Sociologyunassessed73.1
MMLU: US Foreign Policyunassessed81.0
MMLU: Virologyunassessed49.4
MMLU: World Religionsunassessed83.6
MuSRunassessed39.7
TruthfulQAunassessed68.3
WinoGrandeunassessed77.2

Data

This card as JSON · Edit on GitHub