Updated minutes ago
WizardLM

WizardMath 70B

Microsoft · dense · 70B parameters · 4,096 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

70B

Context Window

4K tokens

Architecture

Dense

Best GPU

H200 SXM

Intelligence Brief

WizardMath 70B is a 70B parameter DENSE model from Microsoft, featuring Grouped Query Attention (GQA) with 80 layers and 8,192 hidden dimensions. With a 4,096 token context window, it supports math, reasoning. For self-hosted inference, H200 SXM delivers optimal throughput at $2553/month.

Architecture Details

TypeDENSE
Total Parameters70B
Active Parameters70B
Layers80
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size32,000

Memory Requirements

BF16 Weights

140.0 GB

FP8 Weights

70.0 GB

INT4 Weights

35.0 GB

KV-Cache per Token327680 bytes
Activation Estimate2.50 GB

Fits on (single GPU) — most practical first

BEST FITRTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4RTX 6000 Ada48GBINT4Radeon PRO W790048GBINT4L2048GBINT4Instinct MI21064GBINT4A1664GBINT4H100 SXM80GBINT4H100 PCIe80GBINT4A100 80GB SXM80GBINT4

GPU Compatibility Matrix

WizardMath 70B is compatible with 38% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
61%
81%
90%
B300288GB
51%
76%
88%
B100 SXM192GB
27%
64%
82%
GB200 NVL72 (per GPU)192GB
27%
64%
82%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
26%
63%
81%
H200 SXM141GB
1%
50%
75%
H2096GB
✗
27%
64%
GH20096GB
✗
27%
64%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
27%
L4048GB
✗
✗
27%
RTX 6000 Ada48GB
✗
✗
27%
L2048GB
✗
✗
27%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
13%
56%
A100 80GB PCIe80GB
✗
13%
56%
A1664GB
✗
✗
45%
RTX A600048GB
✗
✗
27%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H200 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$2553

Cost/M Tokens

$1.73

Use this config →
H20optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

478.0 tok/s

Latency (ITL)

2.1ms

Est. TTFT

0ms

Cost/Month

$940

Cost/M Tokens

$0.75

Use this config →
GH200optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

478.0 tok/s

Latency (ITL)

2.1ms

Est. TTFT

0ms

Cost/Month

$2838

Cost/M Tokens

$2.26

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

H200 SXM

$2553/mo

Min VRAM: 70 GB

Scale

Multi-GPU

H100 SXM x2

560.0 tok/s

TP· $3587/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

H200 SXM
560.0 tok/s
H20
478.0 tok/s
GH200
478.0 tok/s

VRAM Breakdown (H200 SXM, FP8)

Weights
Act
Weights 70.0 GBKV-Cache 2.7 GBActivations 20.0 GBOverhead 3.5 GB

Precision Impact

bf16

140.0 GB

weights/GPU

fp8

70.0 GB

weights/GPU

~560.0 tok/s

int4

35.0 GB

weights/GPU

Capabilities

Features

✗ Tool Use✗ Vision✗ Code✓ Math✓ Reasoning✗ Multilingual✗ Structured Output

Supported Frameworks

vllmsglangtgitensorrt-llm

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy WizardMath 70B

Similar Models

Code Llama 70B

70B params · dense

Quality: 60

from $0.90/M

Larger context, Higher qualityCompare →

Llama 2 70B

70B params · dense

Quality: 62

from $0.90/M

Higher qualityCompare →

Claude Sonnet 4

70B params · dense

Quality: 86

from $15.00/M

Larger context, Higher qualityCompare →

o1-mini

70B params · dense

Quality: 83

from $12.00/M

Larger context, Higher qualityCompare →

o3-mini

70B params · dense

Quality: 86

from $4.40/M

Larger context, Higher qualityCompare →

Frequently Asked Questions

How much VRAM does WizardMath 70B need for inference?

WizardMath 70B requires approximately 140.0 GB of VRAM at BF16 precision, 70.0 GB at FP8, or 35.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (327680 bytes per token) and activations (~2.50 GB).

What is the best GPU for WizardMath 70B?

The top recommended GPU for WizardMath 70B is the H200 SXM using FP8 precision. It achieves approximately 560.0 tokens/sec at an estimated cost of $2553/month ($1.73/M tokens). Score: 100/100.

How much does WizardMath 70B inference cost?

WizardMath 70B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.