Updated minutes ago
Upstage

Solar Pro 22B

Upstage · dense · 22B parameters · 4,096 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

22B

Context Window

4K tokens

Architecture

Dense

Best GPU

H20

Cheapest API

$0.50/M

Intelligence Brief

Solar Pro 22B is a 22B parameter DENSE model from Upstage, featuring Grouped Query Attention (GQA) with 48 layers and 4,096 hidden dimensions. With a 4,096 token context window, it supports tools, structured output, code, math, multilingual. The most cost-effective API deployment is via upstage at $0.50/M output tokens. For self-hosted inference, H20 delivers optimal throughput at $940/month.

Architecture Details

TypeDENSE
Total Parameters22B
Active Parameters22B
Layers48
Hidden Dimension4,096
Attention Heads32
KV Heads8
Head Dimension128
Vocab Size32,000

Memory Requirements

BF16 Weights

44.0 GB

FP8 Weights

22.0 GB

INT4 Weights

11.0 GB

KV-Cache per Token196608 bytes
Activation Estimate1.50 GB

Fits on (single GPU) — most practical first

BEST FITA400016GBINT4RTX 408016GBINT4T416GBINT4V100 16GB16GBINT4RTX 4060 Ti 16GB16GBINT4A216GBINT4TPU v5e16GBINT4A10G24GBINT4A3024GBINT4L424GBINT4RTX 409024GBINT4RTX 309024GBINT4

GPU Compatibility Matrix

Solar Pro 22B is compatible with 74% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
88%
94%
97%
B300288GB
85%
92%
96%
B100 SXM192GB
77%
89%
94%
GB200 NVL72 (per GPU)192GB
77%
89%
94%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
77%
88%
94%
H200 SXM141GB
69%
84%
92%
H2096GB
54%
77%
89%
GH20096GB
54%
77%
89%
Ada Lovelace(11 GPUs)
L40S48GB
8%
54%
77%
L4048GB
8%
54%
77%
RTX 6000 Ada48GB
8%
54%
77%
L2048GB
8%
54%
77%
Ampere(16 GPUs)
A100 80GB SXM80GB
45%
73%
86%
A100 80GB PCIe80GB
45%
73%
86%
A1664GB
31%
66%
83%
RTX A600048GB
8%
54%
77%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H20optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$940

Cost/M Tokens

$0.34

Use this config →
H100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

95/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$0.65

Use this config →
H100 PCIeoptimal

FP8 · 1 GPU · tensorrt-llm

95/100

score

Throughput

760.5 tok/s

Latency (ITL)

1.3ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$0.90

Use this config →

Deployment Options

API

API Deployment

upstage

$0.50/M

output tokens

Self-Hosted

Single GPU

H20

$940/mo

Min VRAM: 22 GB

Scale

Multi-GPU

A100 40GB SXM x2

375.0 tok/s

TP· $1613/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
upstage$0.50$0.50
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
upstageBest Value$0.50$0.50$5

Cost per 1,000 Requests

Short (500 tok)

$0.35

via upstage

Medium (2K tok)

$1.40

via upstage

Long (8K tok)

$5.00

via upstage

Performance Estimates

Throughput by GPU

H20
1.1K tok/s
H100 SXM
1.1K tok/s
H100 PCIe
760.5 tok/s

VRAM Breakdown (H20, FP8)

Weights
Act
Weights 22.0 GBKV-Cache 1.6 GBActivations 12.0 GBOverhead 1.1 GB

Precision Impact

bf16

44.0 GB

weights/GPU

fp8

22.0 GB

weights/GPU

~1.1K tok/s

int4

11.0 GB

weights/GPU

Capabilities

Features

✓ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllmsglangtgitensorrt-llmollama

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Solar Pro 22B

Similar Models

Codestral 22B

22B params · dense

Quality: 63

from $0.90/M

Larger context, Higher quality, More expensiveCompare →

Mistral Small 24B

24B params · dense

Quality: 68

from $0.30/M

Larger context, Higher quality, CheaperCompare →

Mistral Small 3.1 24B

24B params · dense

Quality: 50

from $0.30/M

Larger context, CheaperCompare →

GigaChat 20B

20B params · dense

Quality: 50

Larger contextCompare →

Claude 3.5 Haiku

20B params · dense

Quality: 67

from $4.00/M

Larger context, Higher quality, More expensiveCompare →

Frequently Asked Questions

How much VRAM does Solar Pro 22B need for inference?

Solar Pro 22B requires approximately 44.0 GB of VRAM at BF16 precision, 22.0 GB at FP8, or 11.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (196608 bytes per token) and activations (~1.50 GB).

What is the best GPU for Solar Pro 22B?

The top recommended GPU for Solar Pro 22B is the H20 using FP8 precision. It achieves approximately 1.1K tokens/sec at an estimated cost of $940/month ($0.34/M tokens). Score: 100/100.

How much does Solar Pro 22B inference cost?

Solar Pro 22B API inference starts from $0.50/M input tokens and $0.50/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.