Nous Hermes 2 SOLAR 10.7B

Nous Research · nous-research/nous-hermes-2-solar-10-7b

ProviderNous Research
Familyhermes
Typellm-chat
Statusactive
Released2024-01-01

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed66.7
BIG-Bench Hardunassessed54.1
GPQA Diamondunassessed29.4
GSM8Kunassessed69.4
HellaSwagunassessed84.9
IFEvalunassessed52.8
MATH-500unassessed5.7
MMLU: Abstract Algebraunassessed35.0
MMLU: Anatomyunassessed56.3
MMLU: Astronomyunassessed76.3
MMLU: Business Ethicsunassessed74.0
MMLU: Clinical Knowledgeunassessed69.1
MMLU: College Biologyunassessed72.9
MMLU: College Chemistryunassessed47.0
MMLU: College Computer Scienceunassessed42.0
MMLU: College Mathematicsunassessed37.0
MMLU: College Medicineunassessed64.2
MMLU: College Physicsunassessed41.2
MMLU: Computer Securityunassessed73.0
MMLU: Conceptual Physicsunassessed59.6
MMLU: Econometricsunassessed54.4
MMLU: Electrical Engineeringunassessed57.9
MMLU: Elementary Mathematicsunassessed48.9
MMLU: Formal Logicunassessed45.2
MMLU: Global Factsunassessed32.0
MMLU: High School Biologyunassessed80.6
MMLU: High School Chemistryunassessed51.7
MMLU: High School Computer Scienceunassessed71.0
MMLU: High School European Historyunassessed83.0
MMLU: High School Geographyunassessed87.9
MMLU: High School Government and Politicsunassessed89.6
MMLU: High School Macroeconomicsunassessed67.4
MMLU: High School Mathematicsunassessed36.3
MMLU: High School Microeconomicsunassessed69.3
MMLU: High School Physicsunassessed37.7
MMLU: High School Psychologyunassessed85.1
MMLU: High School Statisticsunassessed52.8
MMLU: High School US Historyunassessed83.8
MMLU: High School World Historyunassessed87.8
MMLU: Human Agingunassessed74.9
MMLU: Human Sexualityunassessed77.9
MMLU: International Lawunassessed81.8
MMLU: Jurisprudenceunassessed78.7
MMLU: Logical Fallaciesunassessed74.8
MMLU: Machine Learningunassessed50.9
MMLU: Managementunassessed80.6
MMLU: Marketingunassessed88.0
MMLU: Medical Geneticsunassessed74.0
MMLU: Miscellaneousunassessed82.8
MMLU: Moral Disputesunassessed74.0
MMLU: Moral Scenariosunassessed34.9
MMLU: Nutritionunassessed77.8
MMLU: Philosophyunassessed72.7
MMLU: Prehistoryunassessed77.5
MMLU-Prounassessed34.6
MMLU: Professional Accountingunassessed51.8
MMLU: Professional Lawunassessed50.1
MMLU: Professional Medicineunassessed75.7
MMLU: Professional Psychologyunassessed68.5
MMLU: Public Relationsunassessed70.0
MMLU: Security Studiesunassessed78.8
MMLU: Sociologyunassessed82.6
MMLU: US Foreign Policyunassessed91.0
MMLU: Virologyunassessed56.0
MMLU: World Religionsunassessed83.0
MuSRunassessed43.7
TruthfulQAunassessed55.8
WinoGrandeunassessed82.8

Data

This card as JSON · Edit on GitHub