Llama 3.1 Nemotron 70B

NVIDIA · nvidia/llama-3-1-nemotron-70b-instruct-hf

Provider	NVIDIA
Family	llama-nemotron
Type	llm-chat
Status	active
Released	2024-10-12
Updated	2025-04-13
Parameters	70.6B
Open weights	yes

Lineage

Relationship	Model
Is a finetune of	meta-llama/Llama-3.1-70B-Instruct

What it fits on

Ordered by predicted speed. Bandwidth sets decode rate; memory decides whether it runs at all.

Every figure here is computed, not measured. Fit is weights at each quantisation against device memory, with a 25% allowance for the KV cache, activations and the OS. Decode rate is the memory-bandwidth roofline at 70% efficiency. Nobody has run this model on these devices.

Device	Memory	Bandwidth	Best quality	Weights	tok/s	Fastest
NVIDIA GB200 Grace Blackwell Superchip	372.0 GB	16000.0 GB/s	bf16	141.11 GB	~79.4	~317.5 q4
AMD Instinct MI355X	288.0 GB	8000.0 GB/s	bf16	141.11 GB	~39.7	~158.7 q4
NVIDIA B200	180.0 GB	7700.0 GB/s	fp8	70.55 GB	~76.4	~152.8 q4
Google TPU7x (Ironwood)	192.0 GB	7380.0 GB/s	bf16	141.11 GB	~36.6	~146.4 q4
AMD Instinct MI325X	256.0 GB	6000.0 GB/s	bf16	141.11 GB	~29.8	~119.1 q4
AMD Instinct MI300X	192.0 GB	5300.0 GB/s	bf16	141.11 GB	~26.3	~105.2 q4
NVIDIA H200 SXM	141.0 GB	4800.0 GB/s	fp8	70.55 GB	~47.6	~95.2 q4
NVIDIA H100 NVL	94.0 GB	3900.0 GB/s	q6	52.92 GB	~51.6	~77.4 q4
NVIDIA H100 SXM	80.0 GB	3350.0 GB/s	q6	52.92 GB	~44.3	~66.5 q4
AMD Instinct MI250X	128.0 GB	3200.0 GB/s	fp8	70.55 GB	~31.7	~63.5 q4
Google TPU v5p	95.0 GB	2765.0 GB/s	fp8	70.55 GB	~27.4	~54.9 q4
NVIDIA A100 80GB SXM	80.0 GB	2039.0 GB/s	q6	52.92 GB	~27.0	~40.5 q4
NVIDIA H100 PCIe	80.0 GB	2000.0 GB/s	q6	52.92 GB	~26.5	~39.7 q4
AMD Instinct MI210	64.0 GB	1600.0 GB/s	q5	44.1 GB	~25.4	~31.7 q4
NVIDIA L40S	48.0 GB	864.0 GB/s	q4	35.28 GB	~17.1	~17.1 q4
Apple M3 Ultra	512.0 GB	819.0 GB/s	bf16	141.11 GB	~4.1	~16.3 q4
Apple M2 Ultra	192.0 GB	800.0 GB/s	bf16	141.11 GB	~4.0	~15.9 q4
Apple M4 Max	128.0 GB	546.0 GB/s	fp8	70.55 GB	~5.4	~10.8 q4
Apple M1 Max	64.0 GB	400.0 GB/s	q5	44.1 GB	~6.3	~7.9 q4
Apple M2 Max	96.0 GB	400.0 GB/s	fp8	70.55 GB	~4.0	~7.9 q4
Apple M3 Max	128.0 GB	400.0 GB/s	fp8	70.55 GB	~4.0	~7.9 q4
Apple M4 Pro	64.0 GB	273.0 GB/s	q5	44.1 GB	~4.3	~5.4 q4

Where it runs

Taken from the card's availability section.

Platform	Id on that platform
Hugging Face	nvidia/Llama-3.1-Nemotron-70B-Instruct-HF

Verified benchmark evidence

Each row was checked against its source by a reviewer, and carries the model identifier as actually evaluated — which is not always the same as this card's.

Benchmark	Evaluated as	Score	Evidence date	Source kind
aa_lcr	Llama 3.1 Nemotron 70B	8.33%	2026-09-10 evaluated	independent evaluator	source
critpt	Llama 3.1 Nemotron 70B	0.0%	2026-09-10 evaluated	independent evaluator	source
gpqa_diamond	Llama 3.1 Nemotron 70B	46.46%	2026-09-10 evaluated	independent evaluator	source

Reported benchmark scores

This card reports no benchmark scores yet.

Benchmark	Catalogue standing	Score

Data

This card as JSON · See it in the graph · Edit on GitHub