Updated minutes ago
ReleasedOctober 30, 2025Verified 1mo ago · huggingface.co
Moonshot

Kimi-Linear-48B-A3B-Base

moonshotai · moe · 48B parameters · 1,048,576 context

Quality
50.0
Deploy this model →Use via API →Calculate ROI →Compare with othersFine-tune this model

Parameters

48B

Context Window

1024K tokens

Architecture

MoE

Best GPU

B200 SXM

Intelligence Brief

Kimi-Linear-48B-A3B-Base is a 48B parameter Mixture-of-Experts (256 experts, 8 active) model from moonshotai, featuring Multi-Head Attention (MHA) with 27 layers and 2,304 hidden dimensions. With a 1,048,576 token context window, it supports code, math, multilingual. For self-hosted inference, B200 SXM delivers optimal throughput at $4261/month.

Recent changes

Loading…

Related models

3 suggestions

Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.

Architecture Details

TypeMOE
Total Parameters48B
Active Parameters3B
Layers27
Hidden Dimension2,304
Attention Heads32
KV Heads32
Head Dimension72
Vocab Size163,840
Total Experts256
Active Experts8

Memory Requirements

BF16 Weights

96.0 GB

FP8 Weights

48.0 GB

INT4 Weights

24.0 GB

KV-Cache per Token64512 bytes
Activation Estimate6.00 GB

Fits on (single GPU) — most practical first

BEST FITRTX 509032GBINT4V100 32GB32GBINT4Instinct MI10032GBINT4TPU v432GBINT4TPU v6e (Trillium)32GBINT4Cloud AI 10032GBINT4A100 40GB SXM40GBINT4A100 40GB PCIe40GBINT4RTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4

GPU Compatibility Matrix

Kimi-Linear-48B-A3B-Base is compatible with 40% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
73%
87%
93%
B300288GB
67%
83%
92%
B100 SXM192GB
50%
75%
88%
GB200 NVL72 (per GPU)192GB
50%
75%
88%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
49%
74%
87%
H200 SXM141GB
32%
66%
83%
H2096GB
✗
50%
75%
GH20096GB
✗
50%
75%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
50%
L4048GB
✗
✗
50%
RTX 6000 Ada48GB
✗
✗
50%
L2048GB
✗
✗
50%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
40%
70%
A100 80GB PCIe80GB
✗
40%
70%
A1664GB
✗
25%
63%
RTX A600048GB
✗
✗
50%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B200 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$4261

Cost/M Tokens

$1.54

Use this config →
B100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$4271

Cost/M Tokens

$1.55

Use this config →
GB200 NVL72 (per GPU)optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$6169

Cost/M Tokens

$2.24

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

B200 SXM

$4261/mo

Min VRAM: 48 GB

Scale

Multi-GPU

A100 80GB SXM x2

1.1K tok/s

TP· $2259/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

B200 SXM
1.1K tok/s
B100 SXM
1.1K tok/s
GB200 NVL72 (per GPU)
1.1K tok/s

VRAM Breakdown (B200 SXM, FP8)

Weights
Act
Weights 48.0 GBKV-Cache 2.0 GBActivations 48.0 GBOverhead 2.4 GB

Precision Impact

bf16

96.0 GB

weights/GPU

fp8

48.0 GB

weights/GPU

~1.1K tok/s

int4

24.0 GB

weights/GPU

Capabilities

Features

✗ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✗ Structured Output

Supported Frameworks

vllmsglang

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Kimi-Linear-48B-A3B-Base

Similar Models

Mixtral 8x7B Instruct

46.7B params · moe

Quality: 69

from $0.24/M

Smaller context, Higher qualityCompare →

Mixtral 8x7B

46.7B params · moe

Quality: 67

from $0.50/M

Smaller context, Higher qualityCompare →

Amazon Nova Pro

50B params · dense

Quality: 36

from $3.20/M

Smaller context, Lower qualityCompare →

Gemini 1.5 Flash

50B params · moe

Quality: 75

from $0.30/M

Higher qualityCompare →

Gemini 2.0 Flash

50B params · moe

Quality: 80

from $0.40/M

Higher qualityCompare →

Frequently Asked Questions

How much VRAM does Kimi-Linear-48B-A3B-Base need for inference?

Kimi-Linear-48B-A3B-Base requires approximately 96.0 GB of VRAM at BF16 precision, 48.0 GB at FP8, or 24.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (64512 bytes per token) and activations (~6.00 GB).

What is the best GPU for Kimi-Linear-48B-A3B-Base?

The top recommended GPU for Kimi-Linear-48B-A3B-Base is the B200 SXM using FP8 precision. It achieves approximately 1.1K tokens/sec at an estimated cost of $4261/month ($1.54/M tokens). Score: 100/100.

How much does Kimi-Linear-48B-A3B-Base inference cost?

Kimi-Linear-48B-A3B-Base inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.