Updated minutes ago
Mistral Large 2
Mistral AI · dense · 123B parameters · 131,072 context
Quality82.0
Architecture Details
TypeDENSE
Total Parameters123B
Active Parameters123B
Layers88
Hidden Dimension12,288
Attention Heads96
KV Heads8
Head Dimension128
Vocab Size32,768
Memory Requirements
BF16 Weights
246.0 GB
FP8 Weights
123.0 GB
INT4 Weights
61.5 GB
KV-Cache per Token360448 bytes
Activation Estimate3.50 GB
Fits on (single-node)
B200 SXM FP8B100 SXM FP8GB200 NVL72 (per GPU) FP8GB300 NVL72 (per GPU) FP8H200 SXM INT4H100 SXM INT4H100 PCIe INT4H100 NVL INT4
GPU Recommendations
B200 SXMoptimal
FP8 · 1 GPU · tensorrt-llm
100/100
score
Throughput
280.0 tok/s
Cost/Month
$4261
Cost/M Tokens
$5.79
B100 SXMoptimal
FP8 · 1 GPU · tensorrt-llm
100/100
score
Throughput
280.0 tok/s
Cost/Month
$4271
Cost/M Tokens
$5.80
GB200 NVL72 (per GPU)optimal
FP8 · 1 GPU · tensorrt-llm
100/100
score
Throughput
280.0 tok/s
Cost/Month
$6169
Cost/M Tokens
$8.38
API Pricing Comparison
| Provider | Input $/M | Output $/M | Badges |
|---|---|---|---|
| together | $2.50 | $2.50 | Cheapest |
| mistral | $2.00 | $6.00 | Low Input |
Quality Benchmarks
MMLU84.0
HumanEval53.0
GSM8K91.2
MT-Bench84.0
Capabilities
Features
✓ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output
Supported Frameworks
vllmsglangtgitensorrt-llm
Supported Precisions
BF16 (default)FP8INT4