Updated minutes ago
Snowflake

Snowflake Arctic 128x3B

Snowflake · moe · 395B parameters · 4,096 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

395B

Context Window

4K tokens

Architecture

MoE

Best GPU

B200 SXM

Intelligence Brief

Snowflake Arctic 128x3B is a 395B parameter Mixture-of-Experts (128 experts, 2 active) model from Snowflake, featuring Grouped Query Attention (GQA) with 35 layers and 7,168 hidden dimensions. With a 4,096 token context window, it supports structured output, code. For self-hosted inference, B200 SXM delivers optimal throughput at $17044/month.

Architecture Details

TypeMOE
Total Parameters395B
Active Parameters17B
Layers35
Hidden Dimension7,168
Attention Heads56
KV Heads8
Head Dimension128
Vocab Size32,000
Total Experts128
Active Experts2

Memory Requirements

BF16 Weights

790.0 GB

FP8 Weights

395.0 GB

INT4 Weights

197.5 GB

KV-Cache per Token143360 bytes
Activation Estimate2.00 GB

Fits on (single GPU) — most practical first

BEST FITInstinct MI325X256GBINT4B300288GBINT4B200 NVL (pair)360GBINT4

Fits on (multi-GPU with Tensor Parallelism)

RECOMMENDEDx2Instinct MI250X128GB eachINT4x2Gaudi 3128GB eachINT4x2Gaudi 3 HL-325L128GB eachINT4x2H200 SXM141GB eachINT4x2B200 SXM180GB eachINT4x2H100 NVL 94GB (per GPU pair)188GB eachINT4x2B100 SXM192GB eachINT4x2GB200 NVL72 (per GPU)192GB eachINT4x2GB300 NVL72 (per GPU)192GB eachINT4

Multi-GPU configurations use Tensor Parallelism (TP) to split model layers across GPUs. Requires NVLink or NVSwitch interconnect for optimal performance.

GPU Compatibility Matrix

Snowflake Arctic 128x3B is compatible with 2% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
✗
✗
45%
B300288GB
✗
✗
31%
B100 SXM192GB
✗
✗
✗
GB200 NVL72 (per GPU)192GB
✗
✗
✗
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
✗
✗
✗
H200 SXM141GB
✗
✗
✗
H2096GB
✗
✗
✗
GH20096GB
✗
✗
✗
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
✗
L4048GB
✗
✗
✗
RTX 6000 Ada48GB
✗
✗
✗
L2048GB
✗
✗
✗
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
✗
✗
A100 80GB PCIe80GB
✗
✗
✗
A1664GB
✗
✗
✗
RTX A600048GB
✗
✗
✗
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B200 SXMoptimal

FP8 · 4 GPUs · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$17044

Cost/M Tokens

$23.16

Use this config →
B100 SXMoptimal

FP8 · 4 GPUs · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$17082

Cost/M Tokens

$23.21

Use this config →
H200 SXMoptimal

FP8 · 4 GPUs · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$10211

Cost/M Tokens

$13.88

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

Requires multi-GPU setup (395 GB VRAM needed)

Scale

Multi-GPU

B200 SXM x4

280.0 tok/s

TP· $17044/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

B200 SXM
280.0 tok/s
B100 SXM
280.0 tok/s
H200 SXM
280.0 tok/s

VRAM Breakdown (B200 SXM, FP8)

Weights
Weights 98.8 GBKV-Cache 1.2 GBActivations 16.0 GBOverhead 4.9 GB

Precision Impact

bf16

197.5 GB

weights/GPU

fp8

98.8 GB

weights/GPU

~280.0 tok/s

int4

49.4 GB

weights/GPU

Capabilities

Features

✗ Tool Use✗ Vision✓ Code✗ Math✗ Reasoning✗ Multilingual✓ Structured Output

Supported Frameworks

vllmsglang

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Snowflake Arctic 128x3B

Similar Models

Snowflake Arctic 480B

480B params · moe

Quality: 50

from $1.50/M

Similar specsCompare →

Jamba 1.5 Large

398B params · hybrid

Quality: 50

from $8.00/M

Larger contextCompare →

Llama 4 Maverick

400B params · moe

Quality: 84

from $1.80/M

Larger context, Higher qualityCompare →

Llama 3.1 405B

405B params · dense

Quality: 81

from $3.00/M

Larger context, Higher qualityCompare →

Nemotron 340B

340B params · dense

Quality: 85

from $4.20/M

Larger context, Higher qualityCompare →

Frequently Asked Questions

How much VRAM does Snowflake Arctic 128x3B need for inference?

Snowflake Arctic 128x3B requires approximately 790.0 GB of VRAM at BF16 precision, 395.0 GB at FP8, or 197.5 GB at INT4 quantization. Additional VRAM is needed for KV-cache (143360 bytes per token) and activations (~2.00 GB).

What is the best GPU for Snowflake Arctic 128x3B?

The top recommended GPU for Snowflake Arctic 128x3B is the B200 SXM (x4) using FP8 precision. It achieves approximately 280.0 tokens/sec at an estimated cost of $17044/month ($23.16/M tokens). Score: 100/100.

How much does Snowflake Arctic 128x3B inference cost?

Snowflake Arctic 128x3B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.