InternLM 2.5 20B
Shanghai AI Lab · dense · 19.9B parameters · 262,144 context
Parameters
19.9B
Context Window
256K tokens
Architecture
Dense
Best GPU
H100 SXM
Cheapest API
$0.50/M
Intelligence Brief
InternLM 2.5 20B is a 19.9B parameter DENSE model from Shanghai AI Lab, featuring Grouped Query Attention (GQA) with 48 layers and 6,144 hidden dimensions. With a 262,144 token context window, it supports tools, structured output, code, math, multilingual, reasoning. The most cost-effective API deployment is via internlm at $0.50/M output tokens. For self-hosted inference, H100 SXM delivers optimal throughput at $1794/month.
Provider pricing
1 provider · canonical: internlm| Provider | Input $/M | Output $/M ▲ | Notes |
|---|---|---|---|
| internlmcanonical | $0.500 | $0.500 | cheapest input · cheapest output |
Prices update via the nightly pricing cron + admin approvals at /admin/ingest-queue. The leaderboard's Input/Output cells show the canonical rate above; this table shows the full spread.
Recent changes
Loading…
Related models
4 suggestions- InternLM 2.5 7BInternLM 2.5 · 7.74B$0.200/M out
Yi 1.5 34BYi 1.5 · 34.4B$0.800/M out- Yi 1.5 9BYi 1.5 · 8.83B$0.200/M out
Qwen 2.5 14BQwen 2.5 · 14.8B$0.300/M out
Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.
Architecture Details
Memory Requirements
BF16 Weights
39.8 GB
FP8 Weights
19.9 GB
INT4 Weights
9.9 GB
GPU Compatibility Matrix
InternLM 2.5 20B is compatible with 76% of GPU configurations across 41 GPUs at 3 precision levels.
GPU Recommendations
FP8 · 1 GPU · tensorrt-llm
100/100
score
Throughput
1.1K tok/s
Latency (ITL)
1.0ms
Est. TTFT
0ms
Cost/Month
$1794
Cost/M Tokens
$0.65
FP8 · 1 GPU · tensorrt-llm
100/100
score
Throughput
1.1K tok/s
Latency (ITL)
1.0ms
Est. TTFT
0ms
Cost/Month
$940
Cost/M Tokens
$0.34
FP8 · 1 GPU · tensorrt-llm
95/100
score
Throughput
840.8 tok/s
Latency (ITL)
1.2ms
Est. TTFT
0ms
Cost/Month
$1794
Cost/M Tokens
$0.81
Deployment Options
API Deployment
internlm
$0.50/M
output tokens
Single GPU
H100 SXM
$1794/mo
Min VRAM: 20 GB
Multi-GPU
A100 40GB SXM x2
414.2 tok/s
TP· $1613/mo
API Pricing Comparison
| Provider | Input $/M | Output $/M | Badges |
|---|---|---|---|
| internlm | $0.50 | $0.50 | Cheapest |
Cost Analysis
| Provider | Input $/M | Output $/M | ~Monthly Cost |
|---|---|---|---|
| internlmBest Value | $0.50 | $0.50 | $5 |
Cost per 1,000 Requests
Short (500 tok)
$0.35
via internlm
Medium (2K tok)
$1.40
via internlm
Long (8K tok)
$5.00
via internlm
Performance Estimates
Throughput by GPU
VRAM Breakdown (H100 SXM, FP8)
Precision Impact
bf16
39.8 GB
weights/GPU
fp8
19.9 GB
weights/GPU
~1.1K tok/s
int4
9.9 GB
weights/GPU
Capabilities
Features
Supported Frameworks
Supported Precisions
Where to Deploy InternLM 2.5 20B
Self-Hosted Infrastructure
Similar Models
InternLM 2.5 7B
7.74B params · dense
Quality: 50
from $0.20/M
Claude 3.5 Haiku
20B params · dense
Quality: 67
from $4.00/M
GPT-3.5 Turbo
20B params · dense
Quality: 67
from $1.50/M
GigaChat 20B
20B params · dense
Quality: 50
InternLM 20B
20B params · dense
Quality: 50
Frequently Asked Questions
How much VRAM does InternLM 2.5 20B need for inference?
InternLM 2.5 20B requires approximately 39.8 GB of VRAM at BF16 precision, 19.9 GB at FP8, or 9.9 GB at INT4 quantization. Additional VRAM is needed for KV-cache (196608 bytes per token) and activations (~1.50 GB).
What is the best GPU for InternLM 2.5 20B?
The top recommended GPU for InternLM 2.5 20B is the H100 SXM using FP8 precision. It achieves approximately 1.1K tokens/sec at an estimated cost of $1794/month ($0.65/M tokens). Score: 100/100.
How much does InternLM 2.5 20B inference cost?
InternLM 2.5 20B API inference starts from $0.50/M input tokens and $0.50/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.