Nous Hermes 2 Mixtral 8x7B DPO

Nous Research · nous-research/nous-hermes-2-mixtral-8x7b-dpo

ProviderNous Research
Familyhermes
Typellm-chat
Statusactive
Released2024-01-11

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed71.1
BIG-Bench Hardunassessed55.4
GPQA Diamondunassessed32.1
GSM8Kunassessed71.6
HellaSwagunassessed87.3
IFEvalunassessed59.0
MATH-500unassessed12.2
MMLU: Abstract Algebraunassessed43.0
MMLU: Anatomyunassessed68.9
MMLU: Astronomyunassessed80.3
MMLU: Business Ethicsunassessed73.0
MMLU: Clinical Knowledgeunassessed79.2
MMLU: College Biologyunassessed83.3
MMLU: College Chemistryunassessed55.0
MMLU: College Computer Scienceunassessed66.0
MMLU: College Mathematicsunassessed43.0
MMLU: College Medicineunassessed69.4
MMLU: College Physicsunassessed49.0
MMLU: Computer Securityunassessed81.0
MMLU: Conceptual Physicsunassessed69.4
MMLU: Econometricsunassessed64.9
MMLU: Electrical Engineeringunassessed69.0
MMLU: Elementary Mathematicsunassessed50.3
MMLU: Formal Logicunassessed57.1
MMLU: Global Factsunassessed45.0
MMLU: High School Biologyunassessed85.2
MMLU: High School Chemistryunassessed59.1
MMLU: High School Computer Scienceunassessed75.0
MMLU: High School European Historyunassessed78.2
MMLU: High School Geographyunassessed85.4
MMLU: High School Government and Politicsunassessed93.8
MMLU: High School Macroeconomicsunassessed69.0
MMLU: High School Mathematicsunassessed34.8
MMLU: High School Microeconomicsunassessed79.4
MMLU: High School Physicsunassessed45.7
MMLU: High School Psychologyunassessed89.5
MMLU: High School Statisticsunassessed66.2
MMLU: High School US Historyunassessed86.8
MMLU: High School World Historyunassessed89.5
MMLU: Human Agingunassessed76.2
MMLU: Human Sexualityunassessed87.8
MMLU: International Lawunassessed86.0
MMLU: Jurisprudenceunassessed84.3
MMLU: Logical Fallaciesunassessed80.4
MMLU: Machine Learningunassessed54.5
MMLU: Managementunassessed85.4
MMLU: Marketingunassessed90.2
MMLU: Medical Geneticsunassessed80.0
MMLU: Miscellaneousunassessed88.0
MMLU: Moral Disputesunassessed80.1
MMLU: Moral Scenariosunassessed57.2
MMLU: Nutritionunassessed80.7
MMLU: Philosophyunassessed79.1
MMLU: Prehistoryunassessed85.5
MMLU-Prounassessed36.7
MMLU: Professional Accountingunassessed52.8
MMLU: Professional Lawunassessed55.5
MMLU: Professional Medicineunassessed78.7
MMLU: Professional Psychologyunassessed78.8
MMLU: Public Relationsunassessed69.1
MMLU: Security Studiesunassessed81.2
MMLU: Sociologyunassessed87.1
MMLU: US Foreign Policyunassessed91.0
MMLU: Virologyunassessed50.6
MMLU: World Religionsunassessed89.5
MuSRunassessed46.0
TruthfulQAunassessed54.8
WinoGrandeunassessed83.1

Data

This card as JSON · Edit on GitHub