Updated minutes ago
01.AI

Yi-Large

01.AI · moe · 102.6B parameters · 32,768 context

Quality
74.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

102.6B

Context Window

32K tokens

Architecture

MoE

Best GPU

B200 SXM

Cheapest API

$3.00/M

Quality Score

74/100

Intelligence Brief

Yi-Large is a 102.6B parameter Mixture-of-Experts (32 experts, 4 active) model from 01.AI, featuring Grouped Query Attention (GQA) with 64 layers and 8,192 hidden dimensions. With a 32,768 token context window, it supports tools, structured output, code, math, multilingual. On standardized benchmarks, it achieves MMLU 78, HumanEval 47, GSM8K 82. The most cost-effective API deployment is via 01ai at $3.00/M output tokens. For self-hosted inference, B200 SXM delivers optimal throughput at $4261/month.

Architecture Details

TypeMOE
Total Parameters102.6B
Active Parameters24B
Layers64
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size64,000
Total Experts32
Active Experts4

Memory Requirements

BF16 Weights

205.2 GB

FP8 Weights

102.6 GB

INT4 Weights

51.3 GB

KV-Cache per Token262144 bytes
Activation Estimate2.50 GB

Fits on (single GPU) — most practical first

BEST FITInstinct MI21064GBINT4A1664GBINT4H100 SXM80GBINT4H100 PCIe80GBINT4A100 80GB SXM80GBINT4A100 80GB PCIe80GBINT4H100 NVL94GBINT4H2096GBINT4Gaudi 296GBINT4GH20096GBINT4Trainium296GBINT4Instinct MI250X128GBFP8

GPU Compatibility Matrix

Yi-Large is compatible with 21% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
43%
72%
86%
B300288GB
29%
64%
82%
B100 SXM192GB
✗
47%
73%
GB200 NVL72 (per GPU)192GB
✗
47%
73%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
✗
45%
73%
H200 SXM141GB
✗
27%
64%
H2096GB
✗
✗
47%
GH20096GB
✗
✗
47%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
✗
L4048GB
✗
✗
✗
RTX 6000 Ada48GB
✗
✗
✗
L2048GB
✗
✗
✗
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
✗
36%
A100 80GB PCIe80GB
✗
✗
36%
A1664GB
✗
✗
20%
RTX A600048GB
✗
✗
✗
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B200 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$4261

Cost/M Tokens

$5.79

Use this config →
B100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$4271

Cost/M Tokens

$5.80

Use this config →
GB200 NVL72 (per GPU)optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

280.0 tok/s

Latency (ITL)

3.6ms

Est. TTFT

1ms

Cost/Month

$6169

Cost/M Tokens

$8.38

Use this config →

Deployment Options

API

API Deployment

01ai

$3.00/M

output tokens

Self-Hosted

Single GPU

B200 SXM

$4261/mo

Min VRAM: 103 GB

Scale

Multi-GPU

H100 SXM x2

280.0 tok/s

TP· $3587/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
01ai$3.00$3.00
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
01aiBest Value$3.00$3.00$30

Cost per 1,000 Requests

Short (500 tok)

$2.10

via 01ai

Medium (2K tok)

$8.40

via 01ai

Long (8K tok)

$30.00

via 01ai

Performance Estimates

Throughput by GPU

B200 SXM
280.0 tok/s
B100 SXM
280.0 tok/s
GB200 NVL72 (per GPU)
280.0 tok/s

VRAM Breakdown (B200 SXM, FP8)

Weights
Act
Weights 102.6 GBKV-Cache 2.1 GBActivations 20.0 GBOverhead 5.1 GB

Precision Impact

bf16

205.2 GB

weights/GPU

fp8

102.6 GB

weights/GPU

~280.0 tok/s

int4

51.3 GB

weights/GPU

Quality Benchmarks

Above Average
78th percentile across all models
MMLU
78.0
Average (50th pctile)
HumanEval
47.0
Below Average (42th pctile)
GSM8K
82.0
Average (50th pctile)
MT-Bench
80.0
Bottom 25% (0th pctile)

Capabilities

Features

✓ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllmsglang

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Yi-Large

Similar Models

Yi-Lightning

200B params · moe

Quality: 50

from $0.99/M

Lower quality, Cheaper, Larger modelCompare →

Command R+

104B params · dense

Quality: 68

from $2.00/M

Larger context, Lower quality, CheaperCompare →

Inflection 3

100B params · dense

Quality: 74

from $15.00/M

Smaller context, More expensiveCompare →

YaLM 100B

100B params · dense

Quality: 50

Smaller context, Lower qualityCompare →

Llama 4 Scout

109B params · moe

Quality: 73

from $0.30/M

Larger context, CheaperCompare →

Frequently Asked Questions

How much VRAM does Yi-Large need for inference?

Yi-Large requires approximately 205.2 GB of VRAM at BF16 precision, 102.6 GB at FP8, or 51.3 GB at INT4 quantization. Additional VRAM is needed for KV-cache (262144 bytes per token) and activations (~2.50 GB).

What is the best GPU for Yi-Large?

The top recommended GPU for Yi-Large is the B200 SXM using FP8 precision. It achieves approximately 280.0 tokens/sec at an estimated cost of $4261/month ($5.79/M tokens). Score: 100/100.

How much does Yi-Large inference cost?

Yi-Large API inference starts from $3.00/M input tokens and $3.00/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.