Updated minutes ago
Vikhyat

Moondream 2B

Vikhyat · dense · 1.86B parameters · 2,048 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

1.86B

Context Window

2K tokens

Architecture

Dense

Best GPU

RTX 4060

Intelligence Brief

Moondream 2B is a 1.86B parameter DENSE model from Vikhyat, featuring Multi-Head Attention (MHA) with 24 layers and 2,048 hidden dimensions. With a 2,048 token context window, it supports vision. For self-hosted inference, RTX 4060 delivers optimal throughput at $209/month.

Architecture Details

TypeDENSE
Total Parameters1.86B
Active Parameters1.86B
Layers24
Hidden Dimension2,048
Attention Heads16
KV Heads16
Head Dimension128
Vocab Size51,200

Memory Requirements

BF16 Weights

3.7 GB

FP8 Weights

1.9 GB

INT4 Weights

0.9 GB

KV-Cache per Token98304 bytes
Activation Estimate0.20 GB

Fits on (single GPU) — most practical first

BEST FITRTX 40608GBBF16RTX 30708GBBF16RTX 308010GBBF16RTX 4070 Ti12GBBF16RTX 4070 Super12GBBF16RTX 306012GBBF16A400016GBBF16RTX 408016GBBF16T416GBBF16V100 16GB16GBBF16RTX 4060 Ti 16GB16GBBF16A216GBBF16

GPU Compatibility Matrix

Moondream 2B is compatible with 100% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
99%
99%
100%
B300288GB
99%
99%
100%
B100 SXM192GB
98%
99%
100%
GB200 NVL72 (per GPU)192GB
98%
99%
100%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
98%
99%
100%
H200 SXM141GB
97%
99%
99%
H2096GB
96%
98%
99%
GH20096GB
96%
98%
99%
Ada Lovelace(11 GPUs)
L40S48GB
92%
96%
98%
L4048GB
92%
96%
98%
RTX 6000 Ada48GB
92%
96%
98%
L2048GB
92%
96%
98%
Ampere(16 GPUs)
A100 80GB SXM80GB
95%
98%
99%
A100 80GB PCIe80GB
95%
98%
99%
A1664GB
94%
97%
99%
RTX A600048GB
92%
96%
98%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

RTX 4060optimal

BF16 · 1 GPU · vllm

100/100

score

Throughput

394.8 tok/s

Latency (ITL)

2.5ms

Est. TTFT

0ms

Cost/Month

$209

Cost/M Tokens

$0.20

Use this config →
RTX 3070optimal

BF16 · 1 GPU · vllm

100/100

score

Throughput

650.3 tok/s

Latency (ITL)

1.5ms

Est. TTFT

0ms

Cost/Month

$85

Cost/M Tokens

$0.05

Use this config →
A4000optimal

BF16 · 1 GPU · vllm

90/100

score

Throughput

650.3 tok/s

Latency (ITL)

1.5ms

Est. TTFT

0ms

Cost/Month

$161

Cost/M Tokens

$0.09

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

RTX 4060

$209/mo

Min VRAM: 2 GB

Scale

Multi-GPU

RTX 4060

394.8 tok/s

Best available config

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

RTX 4060
394.8 tok/s
RTX 3070
650.3 tok/s
A4000
650.3 tok/s

VRAM Breakdown (RTX 4060, BF16)

Weights
KV
Act
Weights 3.7 GBKV-Cache 3.2 GBActivations 1.6 GBOverhead 0.3 GB

Precision Impact

bf16

3.7 GB

weights/GPU

~394.8 tok/s

fp8

1.9 GB

weights/GPU

int4

0.9 GB

weights/GPU

Capabilities

Features

✗ Tool Use✓ Vision✗ Code✗ Math✗ Reasoning✗ Multilingual✗ Structured Output

Supported Frameworks

ollamavllm

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Moondream 2B

Similar Models

Gemma 3 2B

2B params · dense

Quality: 42

Larger context, Lower qualityCompare →

SmolLM2 1.7B

1.7B params · dense

Quality: 50

Larger contextCompare →

Qwen 3 1.7B

1.7B params · dense

Quality: 50

Larger contextCompare →

Whisper Large V3

1.55B params · dense

Quality: 50

from $0.01/M

Smaller contextCompare →

Qwen 2 VL 2B

2.2B params · dense

Quality: 50

Larger contextCompare →

Frequently Asked Questions

How much VRAM does Moondream 2B need for inference?

Moondream 2B requires approximately 3.7 GB of VRAM at BF16 precision, 1.9 GB at FP8, or 0.9 GB at INT4 quantization. Additional VRAM is needed for KV-cache (98304 bytes per token) and activations (~0.20 GB).

What is the best GPU for Moondream 2B?

The top recommended GPU for Moondream 2B is the RTX 4060 using BF16 precision. It achieves approximately 394.8 tokens/sec at an estimated cost of $209/month ($0.20/M tokens). Score: 100/100.

How much does Moondream 2B inference cost?

Moondream 2B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.