Updated minutes ago
ReleasedJanuary 5, 2024Verified 1mo ago · huggingface.co
DeepSeek

DeepSeek LLM 67B

DeepSeek · dense · 67B parameters · 4,096 context

Quality
66.0
Deploy this model →Use via API →Calculate ROI →Compare with othersFine-tune this model

Parameters

67B

Context Window

4K tokens

Architecture

Dense

Best GPU

H200 SXM

Quality Score

66/100

Intelligence Brief

DeepSeek LLM 67B is a 67B parameter DENSE model from DeepSeek, featuring Grouped Query Attention (GQA) with 95 layers and 8,192 hidden dimensions. With a 4,096 token context window, it supports code, math, multilingual. On standardized benchmarks, it achieves MMLU 71.3, HumanEval 42, GSM8K 72. For self-hosted inference, H200 SXM delivers optimal throughput at $2553/month.

Recent changes

Loading…

Related models

5 suggestions

Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.

Architecture Details

TypeDENSE
Total Parameters67B
Active Parameters67B
Layers95
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size102,400

Memory Requirements

BF16 Weights

134.0 GB

FP8 Weights

67.0 GB

INT4 Weights

33.5 GB

KV-Cache per Token655360 bytes
Activation Estimate2.50 GB

Fits on (single GPU) — most practical first

BEST FITA100 40GB SXM40GBINT4A100 40GB PCIe40GBINT4RTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4RTX 6000 Ada48GBINT4Radeon PRO W790048GBINT4L2048GBINT4Instinct MI21064GBINT4A1664GBINT4H100 SXM80GBFP8

GPU Compatibility Matrix

DeepSeek LLM 67B is compatible with 38% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
63%
81%
91%
B300288GB
53%
77%
88%
B100 SXM192GB
30%
65%
83%
GB200 NVL72 (per GPU)192GB
30%
65%
83%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
29%
64%
82%
H200 SXM141GB
5%
52%
76%
H2096GB
✗
30%
65%
GH20096GB
✗
30%
65%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
30%
L4048GB
✗
✗
30%
RTX 6000 Ada48GB
✗
✗
30%
L2048GB
✗
✗
30%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
16%
58%
A100 80GB PCIe80GB
✗
16%
58%
A1664GB
✗
✗
48%
RTX A600048GB
✗
✗
30%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H200 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$2553

Cost/M Tokens

$1.73

Use this config →
H100 NVLoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

491.7 tok/s

Latency (ITL)

2.0ms

Est. TTFT

0ms

Cost/Month

$2932

Cost/M Tokens

$2.27

Use this config →
H20optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

499.4 tok/s

Latency (ITL)

2.0ms

Est. TTFT

0ms

Cost/Month

$940

Cost/M Tokens

$0.72

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

H200 SXM

$2553/mo

Min VRAM: 67 GB

Scale

Multi-GPU

RTX 3090 x8

258.6 tok/s

TP· $1442/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

H200 SXM
560.0 tok/s
H100 NVL
491.7 tok/s
H20
499.4 tok/s

VRAM Breakdown (H200 SXM, FP8)

Weights
Act
Weights 67.0 GBKV-Cache 3.2 GBActivations 20.0 GBOverhead 3.4 GB

Precision Impact

bf16

134.0 GB

weights/GPU

fp8

67.0 GB

weights/GPU

~560.0 tok/s

int4

33.5 GB

weights/GPU

Quality Benchmarks

Average
69th percentile across all models
MMLU
71.3
Below Average (33th pctile)
HumanEval
42.0
Below Average (29th pctile)
GSM8K
72.0
Below Average (32th pctile)
MT-Bench
76.0
Bottom 25% (0th pctile)

Capabilities

Features

✗ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✗ Structured Output

Supported Frameworks

vllmsglangtgitensorrt-llm

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy DeepSeek LLM 67B

Similar Models

Claude 3 Sonnet

70B params · dense

Quality: 78

from $15.00/M

Larger context, Higher qualityCompare →

Claude Sonnet 4

70B params · dense

Quality: 86

from $15.00/M

Larger context, Higher qualityCompare →

Code Llama 70B

70B params · dense

Quality: 60

from $0.90/M

Larger context, Lower qualityCompare →

Japanese StableLM 70B

70B params · dense

Quality: 50

Larger context, Lower qualityCompare →

Meditron 70B

70B params · dense

Quality: 50

Lower qualityCompare →

Frequently Asked Questions

How much VRAM does DeepSeek LLM 67B need for inference?

DeepSeek LLM 67B requires approximately 134.0 GB of VRAM at BF16 precision, 67.0 GB at FP8, or 33.5 GB at INT4 quantization. Additional VRAM is needed for KV-cache (655360 bytes per token) and activations (~2.50 GB).

What is the best GPU for DeepSeek LLM 67B?

The top recommended GPU for DeepSeek LLM 67B is the H200 SXM using FP8 precision. It achieves approximately 560.0 tokens/sec at an estimated cost of $2553/month ($1.73/M tokens). Score: 100/100.

How much does DeepSeek LLM 67B inference cost?

DeepSeek LLM 67B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.