Updated minutes ago
AI21

Jamba 1.5 Mini

AI21 · hybrid · 52B parameters · 256,000 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

52B

Context Window

250K tokens

Architecture

Dense

Best GPU

B200 SXM

Cheapest API

$0.40/M

Intelligence Brief

Jamba 1.5 Mini is a 52B parameter HYBRID model from AI21, featuring Grouped Query Attention (GQA) with 32 layers and 4,096 hidden dimensions. With a 256,000 token context window, it supports tools, structured output, code, math, multilingual. The most cost-effective API deployment is via ai21 at $0.40/M output tokens. For self-hosted inference, B200 SXM delivers optimal throughput at $4261/month.

Architecture Details

TypeHYBRID
Total Parameters52B
Active Parameters12B
Layers32
Hidden Dimension4,096
Attention Heads32
KV Heads8
Head Dimension128
Vocab Size65,536

Memory Requirements

BF16 Weights

104.0 GB

FP8 Weights

52.0 GB

INT4 Weights

26.0 GB

KV-Cache per Token65536 bytes
Activation Estimate1.50 GB

Fits on (single GPU) — most practical first

BEST FITRTX 509032GBINT4V100 32GB32GBINT4Instinct MI10032GBINT4TPU v432GBINT4TPU v6e (Trillium)32GBINT4Cloud AI 10032GBINT4A100 40GB SXM40GBINT4A100 40GB PCIe40GBINT4RTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4

GPU Compatibility Matrix

Jamba 1.5 Mini is compatible with 40% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
71%
86%
93%
B300288GB
64%
82%
91%
B100 SXM192GB
46%
73%
86%
GB200 NVL72 (per GPU)192GB
46%
73%
86%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
45%
72%
86%
H200 SXM141GB
26%
63%
82%
H2096GB
✗
46%
73%
GH20096GB
✗
46%
73%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
46%
L4048GB
✗
✗
46%
RTX 6000 Ada48GB
✗
✗
46%
L2048GB
✗
✗
46%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
35%
68%
A100 80GB PCIe80GB
✗
35%
68%
A1664GB
✗
19%
59%
RTX A600048GB
✗
✗
46%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B200 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$4261

Cost/M Tokens

$2.90

Use this config →
B100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$4271

Cost/M Tokens

$2.90

Use this config →
GB200 NVL72 (per GPU)optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$6169

Cost/M Tokens

$4.19

Use this config →

Deployment Options

API

API Deployment

ai21

$0.40/M

output tokens

Self-Hosted

Single GPU

B200 SXM

$4261/mo

Min VRAM: 52 GB

Scale

Multi-GPU

A100 80GB SXM x2

560.0 tok/s

TP· $2259/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
ai21$0.20$0.40
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
ai21Best Value$0.20$0.40$3

Cost per 1,000 Requests

Short (500 tok)

$0.18

via ai21

Medium (2K tok)

$0.72

via ai21

Long (8K tok)

$2.40

via ai21

Performance Estimates

Throughput by GPU

B200 SXM
560.0 tok/s
B100 SXM
560.0 tok/s
GB200 NVL72 (per GPU)
560.0 tok/s

VRAM Breakdown (B200 SXM, FP8)

Weights
Act
Weights 52.0 GBKV-Cache 1.1 GBActivations 12.0 GBOverhead 2.6 GB

Precision Impact

bf16

104.0 GB

weights/GPU

fp8

52.0 GB

weights/GPU

~560.0 tok/s

Capabilities

Features

✓ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllmsglang

Supported Precisions

BF16 (default)FP8

Where to Deploy Jamba 1.5 Mini

Similar Models

Jamba Instruct

52B params · moe

Quality: 66

from $0.70/M

Higher quality, More expensiveCompare →

Llama 3.1 Nemotron 51B

51B params · dense

Quality: 78

from $0.40/M

Higher qualityCompare →

Amazon Nova Pro

50B params · dense

Quality: 50

from $3.20/M

More expensiveCompare →

Gemini 2.0 Flash

50B params · moe

Quality: 80

from $0.40/M

Larger context, Higher qualityCompare →

Gemini 1.5 Flash

50B params · moe

Quality: 75

from $0.30/M

Larger context, Higher qualityCompare →

Frequently Asked Questions

How much VRAM does Jamba 1.5 Mini need for inference?

Jamba 1.5 Mini requires approximately 104.0 GB of VRAM at BF16 precision, 52.0 GB at FP8, or 26.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (65536 bytes per token) and activations (~1.50 GB).

What is the best GPU for Jamba 1.5 Mini?

The top recommended GPU for Jamba 1.5 Mini is the B200 SXM using FP8 precision. It achieves approximately 560.0 tokens/sec at an estimated cost of $4261/month ($2.90/M tokens). Score: 100/100.

How much does Jamba 1.5 Mini inference cost?

Jamba 1.5 Mini API inference starts from $0.20/M input tokens and $0.40/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.