Hermes 2 Pro Llama 3 8B

Nous Research · nous-research/hermes-2-pro-llama-3-8b

ProviderNous Research
Familyllama
Typellm-chat
Statusactive
Released2024-04-30

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed63.5
BIG-Bench Hardunassessed50.7
GPQA Diamondunassessed29.3
GSM8Kunassessed67.9
HellaSwagunassessed83.2
IFEvalunassessed53.6
MATH-500unassessed8.4
MMLU: Abstract Algebraunassessed36.0
MMLU: Anatomyunassessed63.0
MMLU: Astronomyunassessed69.7
MMLU: Business Ethicsunassessed63.0
MMLU: Clinical Knowledgeunassessed73.6
MMLU: College Biologyunassessed72.2
MMLU: College Chemistryunassessed41.0
MMLU: College Computer Scienceunassessed47.0
MMLU: College Mathematicsunassessed36.0
MMLU: College Medicineunassessed61.8
MMLU: College Physicsunassessed46.1
MMLU: Computer Securityunassessed74.0
MMLU: Conceptual Physicsunassessed56.6
MMLU: Econometricsunassessed50.9
MMLU: Electrical Engineeringunassessed60.0
MMLU: Elementary Mathematicsunassessed42.9
MMLU: Formal Logicunassessed47.6
MMLU: Global Factsunassessed42.0
MMLU: High School Biologyunassessed80.6
MMLU: High School Chemistryunassessed47.8
MMLU: High School Computer Scienceunassessed66.0
MMLU: High School European Historyunassessed74.5
MMLU: High School Geographyunassessed78.8
MMLU: High School Government and Politicsunassessed88.1
MMLU: High School Macroeconomicsunassessed62.1
MMLU: High School Mathematicsunassessed38.5
MMLU: High School Microeconomicsunassessed72.7
MMLU: High School Physicsunassessed39.1
MMLU: High School Psychologyunassessed82.9
MMLU: High School Statisticsunassessed50.9
MMLU: High School US Historyunassessed81.4
MMLU: High School World Historyunassessed84.8
MMLU: Human Agingunassessed70.4
MMLU: Human Sexualityunassessed76.3
MMLU: International Lawunassessed78.5
MMLU: Jurisprudenceunassessed73.1
MMLU: Logical Fallaciesunassessed71.8
MMLU: Machine Learningunassessed54.5
MMLU: Managementunassessed80.6
MMLU: Marketingunassessed87.6
MMLU: Medical Geneticsunassessed78.0
MMLU: Miscellaneousunassessed83.1
MMLU: Moral Disputesunassessed71.1
MMLU: Moral Scenariosunassessed43.7
MMLU: Nutritionunassessed71.2
MMLU: Philosophyunassessed71.1
MMLU: Prehistoryunassessed71.3
MMLU-Prounassessed30.5
MMLU: Professional Accountingunassessed50.7
MMLU: Professional Lawunassessed46.8
MMLU: Professional Medicineunassessed69.5
MMLU: Professional Psychologyunassessed69.8
MMLU: Public Relationsunassessed61.8
MMLU: Security Studiesunassessed75.5
MMLU: Sociologyunassessed86.6
MMLU: US Foreign Policyunassessed84.0
MMLU: Virologyunassessed53.0
MMLU: World Religionsunassessed81.9
MuSRunassessed42.6
TruthfulQAunassessed56.6
WinoGrandeunassessed76.4

Data

This card as JSON · Edit on GitHub