falcon 40B

TII · tii/falcon-40b

ProviderTII
Familyfalcon
Typellm-code
Statusactive
Released2023-05-24

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed61.9
BIG-Bench Hardunassessed40.2
GPQA Diamondunassessed27.3
HellaSwagunassessed85.3
IFEvalunassessed25.0
MATH-500unassessed1.8
MMLU: Abstract Algebraunassessed34.0
MMLU: Anatomyunassessed53.3
MMLU: Astronomyunassessed57.2
MMLU: Business Ethicsunassessed54.0
MMLU: Clinical Knowledgeunassessed60.8
MMLU: College Biologyunassessed66.7
MMLU: College Chemistryunassessed45.0
MMLU: College Computer Scienceunassessed50.0
MMLU: College Mathematicsunassessed39.0
MMLU: College Medicineunassessed52.0
MMLU: College Physicsunassessed27.5
MMLU: Computer Securityunassessed63.0
MMLU: Conceptual Physicsunassessed41.3
MMLU: Econometricsunassessed33.3
MMLU: Electrical Engineeringunassessed53.1
MMLU: Elementary Mathematicsunassessed33.9
MMLU: Formal Logicunassessed30.2
MMLU: Global Factsunassessed32.0
MMLU: High School Biologyunassessed67.1
MMLU: High School Chemistryunassessed44.3
MMLU: High School Computer Scienceunassessed62.0
MMLU: High School European Historyunassessed69.7
MMLU: High School Geographyunassessed74.2
MMLU: High School Government and Politicsunassessed77.2
MMLU: High School Macroeconomicsunassessed55.4
MMLU: High School Mathematicsunassessed31.5
MMLU: High School Microeconomicsunassessed55.5
MMLU: High School Physicsunassessed31.8
MMLU: High School Psychologyunassessed77.6
MMLU: High School Statisticsunassessed47.7
MMLU: High School US Historyunassessed71.6
MMLU: High School World Historyunassessed71.3
MMLU: Human Agingunassessed73.5
MMLU: Human Sexualityunassessed72.5
MMLU: International Lawunassessed65.3
MMLU: Jurisprudenceunassessed69.4
MMLU: Logical Fallaciesunassessed65.6
MMLU: Machine Learningunassessed29.5
MMLU: Managementunassessed78.6
MMLU: Marketingunassessed79.1
MMLU: Medical Geneticsunassessed66.0
MMLU: Miscellaneousunassessed75.9
MMLU: Moral Disputesunassessed62.7
MMLU: Moral Scenariosunassessed27.0
MMLU: Nutritionunassessed67.3
MMLU: Philosophyunassessed66.9
MMLU: Prehistoryunassessed63.3
MMLU-Prounassessed25.0
MMLU: Professional Accountingunassessed43.6
MMLU: Professional Lawunassessed43.2
MMLU: Professional Medicineunassessed61.0
MMLU: Professional Psychologyunassessed56.5
MMLU: Public Relationsunassessed62.7
MMLU: Security Studiesunassessed66.1
MMLU: Sociologyunassessed79.6
MMLU: US Foreign Policyunassessed81.0
MMLU: Virologyunassessed47.6
MMLU: World Religionsunassessed80.7
MuSRunassessed36.3
TruthfulQAunassessed41.7

Data

This card as JSON · Edit on GitHub