deepseek coder 1.3B instruct

DeepSeek · deepseek/deepseek-coder-1-3b-instruct

ProviderDeepSeek
Familydeepseek
Typellm-code
Statusactive
Released2023-10-29

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.
BenchmarkCatalogue standingScore
ARC-Challengeunassessed28.6
GSM8Kunassessed1.1
HellaSwagunassessed39.9
MMLU: Abstract Algebraunassessed28.0
MMLU: Anatomyunassessed29.6
MMLU: Astronomyunassessed19.7
MMLU: Business Ethicsunassessed29.0
MMLU: Clinical Knowledgeunassessed30.2
MMLU: College Biologyunassessed25.7
MMLU: College Chemistryunassessed25.0
MMLU: College Computer Scienceunassessed29.0
MMLU: College Mathematicsunassessed23.0
MMLU: College Medicineunassessed24.3
MMLU: College Physicsunassessed25.5
MMLU: Computer Securityunassessed37.0
MMLU: Conceptual Physicsunassessed31.1
MMLU: Econometricsunassessed24.6
MMLU: Electrical Engineeringunassessed38.6
MMLU: Elementary Mathematicsunassessed25.9
MMLU: Formal Logicunassessed21.4
MMLU: Global Factsunassessed27.0
MMLU: High School Biologyunassessed28.1
MMLU: High School Chemistryunassessed27.6
MMLU: High School Computer Scienceunassessed33.0
MMLU: High School European Historyunassessed29.1
MMLU: High School Geographyunassessed35.9
MMLU: High School Government and Politicsunassessed32.6
MMLU: High School Macroeconomicsunassessed26.9
MMLU: High School Mathematicsunassessed24.4
MMLU: High School Microeconomicsunassessed26.1
MMLU: High School Physicsunassessed30.5
MMLU: High School Psychologyunassessed31.7
MMLU: High School Statisticsunassessed25.5
MMLU: High School US Historyunassessed25.5
MMLU: High School World Historyunassessed30.8
MMLU: Human Agingunassessed14.8
MMLU: Human Sexualityunassessed35.1
MMLU: International Lawunassessed28.9
MMLU: Jurisprudenceunassessed25.0
MMLU: Logical Fallaciesunassessed28.8
MMLU: Machine Learningunassessed32.1
MMLU: Managementunassessed37.9
MMLU: Marketingunassessed33.3
MMLU: Medical Geneticsunassessed21.0
MMLU: Miscellaneousunassessed29.2
MMLU: Moral Disputesunassessed28.0
MMLU: Moral Scenariosunassessed27.3
MMLU: Nutritionunassessed30.4
MMLU: Philosophyunassessed29.3
MMLU: Prehistoryunassessed28.7
MMLU: Professional Accountingunassessed24.5
MMLU: Professional Lawunassessed29.0
MMLU: Professional Medicineunassessed41.5
MMLU: Professional Psychologyunassessed24.2
MMLU: Public Relationsunassessed33.6
MMLU: Security Studiesunassessed25.3
MMLU: Sociologyunassessed33.8
MMLU: US Foreign Policyunassessed32.0
MMLU: Virologyunassessed25.3
MMLU: World Religionsunassessed21.6
TruthfulQAunassessed44.0
WinoGrandeunassessed52.4

Data

This card as JSON · Edit on GitHub