Updated minutes ago
Claude 3 Sonnet
Anthropic · dense · 70B parameters · 200,000 context
Quality78.0
Architecture Details
TypeDENSE
Total Parameters70B
Active Parameters70B
Layers64
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size152,064
Memory Requirements
BF16 Weights
140.0 GB
FP8 Weights
70.0 GB
INT4 Weights
35.0 GB
KV-Cache per Token131072 bytes
Activation Estimate3.00 GB
Fits on (single-node)
B200 SXM BF16B100 SXM BF16GB200 NVL72 (per GPU) BF16GB300 NVL72 (per GPU) BF16H200 SXM FP8H100 SXM INT4H100 PCIe INT4H100 NVL FP8
GPU Recommendations
B100 SXMoptimal
BF16 · 1 GPU · tensorrt-llm
98/100
score
Throughput
492.8 tok/s
Cost/Month
$4271
Cost/M Tokens
$3.30
GB200 NVL72 (per GPU)optimal
BF16 · 1 GPU · tensorrt-llm
98/100
score
Throughput
492.8 tok/s
Cost/Month
$6169
Cost/M Tokens
$4.76
H20optimal
BF16 · 2 GPUs · tensorrt-llm
95/100
score
Throughput
450.7 tok/s
Cost/Month
$1879
Cost/M Tokens
$1.59
API Pricing Comparison
| Provider | Input $/M | Output $/M | Badges |
|---|---|---|---|
| anthropic | $3.00 | $15.00 | Cheapest |
Quality Benchmarks
MMLU79.0
HumanEval55.0
GSM8K85.0
MT-Bench83.0
Capabilities
Features
✓ Tool Use✓ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output
Supported Frameworks
Supported Precisions
BF16 (default)