Hermes 2 Theta Llama 3 8B

Nous Research · nous-research/hermes-2-theta-llama-3-8b

ProviderNous Research
Familyllama
Typellm-chat
Statusactive
Released2024-05-05

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed63.1
BIG-Bench Hardunassessed52.1
GPQA Diamondunassessed30.4
GSM8Kunassessed70.4
HellaSwagunassessed82.6
IFEvalunassessed65.2
MATH-500unassessed9.7
MMLU: Abstract Algebraunassessed38.0
MMLU: Anatomyunassessed63.7
MMLU: Astronomyunassessed69.1
MMLU: Business Ethicsunassessed64.0
MMLU: Clinical Knowledgeunassessed74.3
MMLU: College Biologyunassessed74.3
MMLU: College Chemistryunassessed48.0
MMLU: College Computer Scienceunassessed49.0
MMLU: College Mathematicsunassessed35.0
MMLU: College Medicineunassessed66.5
MMLU: College Physicsunassessed48.0
MMLU: Computer Securityunassessed79.0
MMLU: Conceptual Physicsunassessed55.7
MMLU: Econometricsunassessed55.3
MMLU: Electrical Engineeringunassessed61.4
MMLU: Elementary Mathematicsunassessed45.2
MMLU: Formal Logicunassessed48.4
MMLU: Global Factsunassessed39.0
MMLU: High School Biologyunassessed81.3
MMLU: High School Chemistryunassessed51.7
MMLU: High School Computer Scienceunassessed70.0
MMLU: High School European Historyunassessed75.8
MMLU: High School Geographyunassessed81.8
MMLU: High School Government and Politicsunassessed88.6
MMLU: High School Macroeconomicsunassessed65.1
MMLU: High School Mathematicsunassessed38.1
MMLU: High School Microeconomicsunassessed75.2
MMLU: High School Physicsunassessed45.0
MMLU: High School Psychologyunassessed83.1
MMLU: High School Statisticsunassessed50.9
MMLU: High School US Historyunassessed83.8
MMLU: High School World Historyunassessed86.9
MMLU: Human Agingunassessed72.6
MMLU: Human Sexualityunassessed80.9
MMLU: International Lawunassessed78.5
MMLU: Jurisprudenceunassessed75.9
MMLU: Logical Fallaciesunassessed73.6
MMLU: Machine Learningunassessed55.4
MMLU: Managementunassessed80.6
MMLU: Marketingunassessed89.3
MMLU: Medical Geneticsunassessed79.0
MMLU: Miscellaneousunassessed84.4
MMLU: Moral Disputesunassessed74.9
MMLU: Moral Scenariosunassessed43.9
MMLU: Nutritionunassessed73.5
MMLU: Philosophyunassessed73.6
MMLU: Prehistoryunassessed73.8
MMLU-Prounassessed33.7
MMLU: Professional Accountingunassessed52.1
MMLU: Professional Lawunassessed48.5
MMLU: Professional Medicineunassessed69.5
MMLU: Professional Psychologyunassessed70.3
MMLU: Public Relationsunassessed65.5
MMLU: Security Studiesunassessed77.1
MMLU: Sociologyunassessed87.6
MMLU: US Foreign Policyunassessed86.0
MMLU: Virologyunassessed54.8
MMLU: World Religionsunassessed80.7
MuSRunassessed39.5
TruthfulQAunassessed55.7
WinoGrandeunassessed76.6

Data

This card as JSON · Edit on GitHub