deepseek coder 6.7B instruct

DeepSeek · deepseek/deepseek-coder-6-7b-instruct

Provider	DeepSeek
Family	deepseek
Type	llm-code
Status	active
Released	2023-10-29

Reported benchmark scores

These scores carry one collection date for the whole card (2024-07) and a source list rather than a source per score, so they cannot be attributed individually. They are shown as reported and are not verified evidence. Benchmark pages carry the reviewed, dated evidence where it exists.

Benchmark	Catalogue standing	Score
ARC-Challenge	unassessed	38.1
GSM8K	unassessed	26.8
HellaSwag	unassessed	55.1
MMLU: Abstract Algebra	unassessed	33.0
MMLU: Anatomy	unassessed	34.8
MMLU: Astronomy	unassessed	33.6
MMLU: Business Ethics	unassessed	42.0
MMLU: Clinical Knowledge	unassessed	41.1
MMLU: College Biology	unassessed	33.3
MMLU: College Chemistry	unassessed	36.0
MMLU: College Computer Science	unassessed	43.0
MMLU: College Mathematics	unassessed	34.0
MMLU: College Medicine	unassessed	33.5
MMLU: College Physics	unassessed	23.5
MMLU: Computer Security	unassessed	61.0
MMLU: Conceptual Physics	unassessed	34.9
MMLU: Econometrics	unassessed	30.7
MMLU: Electrical Engineering	unassessed	44.1
MMLU: Elementary Mathematics	unassessed	34.1
MMLU: Formal Logic	unassessed	36.5
MMLU: Global Facts	unassessed	33.0
MMLU: High School Biology	unassessed	41.9
MMLU: High School Chemistry	unassessed	32.5
MMLU: High School Computer Science	unassessed	56.0
MMLU: High School European History	unassessed	37.6
MMLU: High School Geography	unassessed	42.4
MMLU: High School Government and Politics	unassessed	43.0
MMLU: High School Macroeconomics	unassessed	35.6
MMLU: High School Mathematics	unassessed	27.8
MMLU: High School Microeconomics	unassessed	41.2
MMLU: High School Physics	unassessed	33.1
MMLU: High School Psychology	unassessed	41.7
MMLU: High School Statistics	unassessed	36.6
MMLU: High School US History	unassessed	39.7
MMLU: High School World History	unassessed	37.6
MMLU: Human Aging	unassessed	39.0
MMLU: Human Sexuality	unassessed	42.0
MMLU: International Law	unassessed	45.5
MMLU: Jurisprudence	unassessed	37.0
MMLU: Logical Fallacies	unassessed	46.0
MMLU: Machine Learning	unassessed	33.0
MMLU: Management	unassessed	38.8
MMLU: Marketing	unassessed	65.4
MMLU: Medical Genetics	unassessed	41.0
MMLU: Miscellaneous	unassessed	43.7
MMLU: Moral Disputes	unassessed	41.9
MMLU: Moral Scenarios	unassessed	25.8
MMLU: Nutrition	unassessed	39.2
MMLU: Philosophy	unassessed	42.8
MMLU: Prehistory	unassessed	36.4
MMLU: Professional Accounting	unassessed	31.9
MMLU: Professional Law	unassessed	28.4
MMLU: Professional Medicine	unassessed	34.9
MMLU: Professional Psychology	unassessed	35.1
MMLU: Public Relations	unassessed	52.7
MMLU: Security Studies	unassessed	45.3
MMLU: Sociology	unassessed	42.8
MMLU: US Foreign Policy	unassessed	54.0
MMLU: Virology	unassessed	37.3
MMLU: World Religions	unassessed	35.1
TruthfulQA	unassessed	45.6
WinoGrande	unassessed	56.8

Data

This card as JSON · Edit on GitHub