Claude 3.5 Sonnet
Anthropic · moe · 175B parameters · 200,000 context
Parameters
175B
Context Window
195K tokens
Architecture
MoE
Best GPU
B200 SXM
Cheapest API
$15.00/M
Intelligence Brief
Claude 3.5 Sonnet is a 175B parameter Mixture-of-Experts (8 experts, 2 active) model from Anthropic, featuring Grouped Query Attention (GQA) with 60 layers and 8,192 hidden dimensions. With a 200,000 token context window, it supports tools, vision, structured output, code, math, multilingual. The most cost-effective API deployment is via anthropic at $15.00/M output tokens. For self-hosted inference, B200 SXM delivers optimal throughput at $17044/month.
Provider pricing
1 provider · canonical: anthropic| Provider | Input $/M | Output $/M ▲ | Notes |
|---|---|---|---|
| anthropiccanonical | $3.00 | $15.00 | cheapest input · cheapest output |
Prices update via the nightly pricing cron + admin approvals at /admin/ingest-queue. The leaderboard's Input/Output cells show the canonical rate above; this table shows the full spread.
Recent changes
Loading…
Related models
5 suggestions- Claude 3 OpusClaude · 175B$75.00/M out
- Claude 3 SonnetClaude · 70B$15.00/M out
- Claude 3.5 HaikuClaude · 20B$4.00/M out
- Claude Haiku 4.5Claude · 30B$5.00/M out
- Claude Opus 4.1Claude · 75B$75.00/M out
Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.
Architecture Details
Memory Requirements
BF16 Weights
350.0 GB
FP8 Weights
175.0 GB
INT4 Weights
88.0 GB
Fits on (single GPU) — most practical first
GPU Compatibility Matrix
Claude 3.5 Sonnet is compatible with 15% of GPU configurations across 41 GPUs at 3 precision levels.
GPU Recommendations
BF16 · 4 GPUs · tensorrt-llm
93/100
score
Throughput
280.0 tok/s
Latency (ITL)
3.6ms
Est. TTFT
1ms
Cost/Month
$17044
Cost/M Tokens
$23.16
BF16 · 2 GPUs · tensorrt-llm
93/100
score
Throughput
280.0 tok/s
Latency (ITL)
3.6ms
Est. TTFT
1ms
Cost/Month
$19929
Cost/M Tokens
$27.08
BF16 · 4 GPUs · tensorrt-llm
90/100
score
Throughput
280.0 tok/s
Latency (ITL)
3.6ms
Est. TTFT
1ms
Cost/Month
$10211
Cost/M Tokens
$13.88
Deployment Options
API Deployment
anthropic
$15.00/M
output tokens
Single GPU
Requires multi-GPU setup (175 GB VRAM needed)
Multi-GPU
B200 SXM x4
280.0 tok/s
TP· $17044/mo
API Pricing Comparison
| Provider | Input $/M | Output $/M | Badges |
|---|---|---|---|
| anthropic | $3.00 | $15.00 | Cheapest |
Cost Analysis
| Provider | Input $/M | Output $/M | ~Monthly Cost |
|---|---|---|---|
| anthropicBest Value | $3.00 | $15.00 | $90 |
Cost per 1,000 Requests
Short (500 tok)
$4.50
via anthropic
Medium (2K tok)
$18.00
via anthropic
Long (8K tok)
$54.00
via anthropic
Performance Estimates
Throughput by GPU
VRAM Breakdown (B200 SXM, BF16)
Capabilities
Features
Supported Frameworks
Supported Precisions
Where to Deploy Claude 3.5 Sonnet
Self-Hosted Infrastructure
Similar Models
Claude 3 Opus
175B params · dense
Quality: 80
from $75.00/M
Claude Sonnet 4.6
180B params · moe
Quality: 86
from $15.00/M
Claude Opus 4
200B params · dense
Quality: 90
from $75.00/M
Claude Sonnet 4.5
150B params · moe
Quality: 86
from $15.00/M
Claude 3 Sonnet
70B params · dense
Quality: 78
from $15.00/M
Frequently Asked Questions
How much VRAM does Claude 3.5 Sonnet need for inference?
Claude 3.5 Sonnet requires approximately 350.0 GB of VRAM at BF16 precision, 175.0 GB at FP8, or 88.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (163840 bytes per token) and activations (~4.00 GB).
What is the best GPU for Claude 3.5 Sonnet?
The top recommended GPU for Claude 3.5 Sonnet is the B200 SXM (x4) using BF16 precision. It achieves approximately 280.0 tokens/sec at an estimated cost of $17044/month ($23.16/M tokens). Score: 93/100.
How much does Claude 3.5 Sonnet inference cost?
Claude 3.5 Sonnet API inference starts from $3.00/M input tokens and $15.00/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.