Updated minutes ago
Anthropic

Claude 3 Sonnet

Anthropic · dense · 70B parameters · 200,000 context

Quality
78.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

70B

Context Window

195K tokens

Architecture

Dense

Best GPU

B100 SXM

Cheapest API

$15.00/M

Quality Score

78/100

Intelligence Brief

Claude 3 Sonnet is a 70B parameter DENSE model from Anthropic, featuring Grouped Query Attention (GQA) with 64 layers and 8,192 hidden dimensions. With a 200,000 token context window, it supports tools, vision, structured output, code, math, multilingual. On standardized benchmarks, it achieves MMLU 79, HumanEval 55, GSM8K 85. The most cost-effective API deployment is via anthropic at $15.00/M output tokens. For self-hosted inference, B100 SXM delivers optimal throughput at $4271/month.

Architecture Details

TypeDENSE
Total Parameters70B
Active Parameters70B
Layers64
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size152,064

Memory Requirements

BF16 Weights

140.0 GB

FP8 Weights

70.0 GB

INT4 Weights

35.0 GB

KV-Cache per Token131072 bytes
Activation Estimate3.00 GB

Fits on (single GPU) — most practical first

BEST FITRTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4RTX 6000 Ada48GBINT4Radeon PRO W790048GBINT4L2048GBINT4Instinct MI21064GBINT4A1664GBINT4H100 SXM80GBINT4H100 PCIe80GBINT4A100 80GB SXM80GBINT4

GPU Compatibility Matrix

Claude 3 Sonnet is compatible with 38% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
61%
81%
90%
B300288GB
51%
76%
88%
B100 SXM192GB
27%
64%
82%
GB200 NVL72 (per GPU)192GB
27%
64%
82%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
26%
63%
81%
H200 SXM141GB
1%
50%
75%
H2096GB
✗
27%
64%
GH20096GB
✗
27%
64%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
27%
L4048GB
✗
✗
27%
RTX 6000 Ada48GB
✗
✗
27%
L2048GB
✗
✗
27%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
13%
56%
A100 80GB PCIe80GB
✗
13%
56%
A1664GB
✗
✗
45%
RTX A600048GB
✗
✗
27%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B100 SXMoptimal

BF16 · 1 GPU · tensorrt-llm

98/100

score

Throughput

492.8 tok/s

Latency (ITL)

2.0ms

Est. TTFT

0ms

Cost/Month

$4271

Cost/M Tokens

$3.30

Use this config →
GB200 NVL72 (per GPU)optimal

BF16 · 1 GPU · tensorrt-llm

98/100

score

Throughput

492.8 tok/s

Latency (ITL)

2.0ms

Est. TTFT

0ms

Cost/Month

$6169

Cost/M Tokens

$4.76

Use this config →
H20optimal

BF16 · 2 GPUs · tensorrt-llm

95/100

score

Throughput

450.7 tok/s

Latency (ITL)

2.2ms

Est. TTFT

0ms

Cost/Month

$1879

Cost/M Tokens

$1.59

Use this config →

Deployment Options

API

API Deployment

anthropic

$15.00/M

output tokens

Self-Hosted

Single GPU

B100 SXM

$4271/mo

Min VRAM: 70 GB

Scale

Multi-GPU

H20 x2

450.7 tok/s

TP· $1879/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
anthropic$3.00$15.00
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
anthropicBest Value$3.00$15.00$90

Cost per 1,000 Requests

Short (500 tok)

$4.50

via anthropic

Medium (2K tok)

$18.00

via anthropic

Long (8K tok)

$54.00

via anthropic

Performance Estimates

Throughput by GPU

B100 SXM
492.8 tok/s
GB200 NVL72 (per GPU)
492.8 tok/s
H20
450.7 tok/s

VRAM Breakdown (B100 SXM, BF16)

Weights
Weights 140.0 GBKV-Cache 4.3 GBActivations 24.0 GBOverhead 7.0 GB

Quality Benchmarks

Above Average
84th percentile across all models
MMLU
79.0
Average (56th pctile)
HumanEval
55.0
Average (57th pctile)
GSM8K
85.0
Average (55th pctile)
MT-Bench
83.0
Bottom 25% (0th pctile)

Capabilities

Features

✓ Tool Use✓ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

Supported Precisions

BF16 (default)

Where to Deploy Claude 3 Sonnet

Similar Models

Claude Sonnet 4

70B params · dense

Quality: 86

from $15.00/M

Higher qualityCompare →

Claude 3 Opus

175B params · dense

Quality: 80

from $75.00/M

More expensive, Larger modelCompare →

Claude Opus 4

200B params · dense

Quality: 90

from $75.00/M

Higher quality, More expensive, Larger modelCompare →

Code Llama 70B

70B params · dense

Quality: 60

from $0.90/M

Smaller context, Lower quality, CheaperCompare →

Llama 2 70B

70B params · dense

Quality: 62

from $0.90/M

Smaller context, Lower quality, CheaperCompare →

Frequently Asked Questions

How much VRAM does Claude 3 Sonnet need for inference?

Claude 3 Sonnet requires approximately 140.0 GB of VRAM at BF16 precision, 70.0 GB at FP8, or 35.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (131072 bytes per token) and activations (~3.00 GB).

What is the best GPU for Claude 3 Sonnet?

The top recommended GPU for Claude 3 Sonnet is the B100 SXM using BF16 precision. It achieves approximately 492.8 tokens/sec at an estimated cost of $4271/month ($3.30/M tokens). Score: 98/100.

How much does Claude 3 Sonnet inference cost?

Claude 3 Sonnet API inference starts from $3.00/M input tokens and $15.00/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.