Updated minutes ago
Google

Gemma 3 2B

Google · dense · 2B parameters · 8,192 context

Quality
42.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

2B

Context Window

8K tokens

Architecture

Dense

Best GPU

RTX 4060

Quality Score

42/100

Intelligence Brief

Gemma 3 2B is a 2B parameter DENSE model from Google, featuring Grouped Query Attention (GQA) with 26 layers and 2,304 hidden dimensions. With a 8,192 token context window, it supports structured output, code, math, multilingual. On standardized benchmarks, it achieves MMLU 50, HumanEval 22, GSM8K 42. For self-hosted inference, RTX 4060 delivers optimal throughput at $209/month.

Architecture Details

TypeDENSE
Total Parameters2B
Active Parameters2B
Layers26
Hidden Dimension2,304
Attention Heads8
KV Heads4
Head Dimension256
Vocab Size262,144

Memory Requirements

BF16 Weights

4.0 GB

FP8 Weights

2.0 GB

INT4 Weights

1.0 GB

KV-Cache per Token26624 bytes
Activation Estimate0.30 GB

Fits on (single GPU) — most practical first

BEST FITRTX 40608GBBF16RTX 30708GBBF16RTX 308010GBBF16RTX 4070 Ti12GBBF16RTX 4070 Super12GBBF16RTX 306012GBBF16A400016GBBF16RTX 408016GBBF16T416GBBF16V100 16GB16GBBF16RTX 4060 Ti 16GB16GBBF16A216GBBF16

GPU Compatibility Matrix

Gemma 3 2B is compatible with 100% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
99%
99%
100%
B300288GB
99%
99%
100%
B100 SXM192GB
98%
99%
99%
GB200 NVL72 (per GPU)192GB
98%
99%
99%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
98%
99%
99%
H200 SXM141GB
97%
99%
99%
H2096GB
96%
98%
99%
GH20096GB
96%
98%
99%
Ada Lovelace(11 GPUs)
L40S48GB
92%
96%
98%
L4048GB
92%
96%
98%
RTX 6000 Ada48GB
92%
96%
98%
L2048GB
92%
96%
98%
Ampere(16 GPUs)
A100 80GB SXM80GB
95%
98%
99%
A100 80GB PCIe80GB
95%
98%
99%
A1664GB
94%
97%
98%
RTX A600048GB
92%
96%
98%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

RTX 4060optimal

BF16 · 1 GPU · vllm

100/100

score

Throughput

330.5 tok/s

Latency (ITL)

3.0ms

Est. TTFT

1ms

Cost/Month

$209

Cost/M Tokens

$0.24

Use this config →
RTX 3070optimal

BF16 · 1 GPU · vllm

100/100

score

Throughput

544.3 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$85

Cost/M Tokens

$0.06

Use this config →
A4000optimal

BF16 · 1 GPU · vllm

90/100

score

Throughput

544.3 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$161

Cost/M Tokens

$0.11

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

RTX 4060

$209/mo

Min VRAM: 2 GB

Scale

Multi-GPU

RTX 4060

330.5 tok/s

Best available config

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

RTX 4060
330.5 tok/s
RTX 3070
544.3 tok/s
A4000
544.3 tok/s

VRAM Breakdown (RTX 4060, BF16)

Weights
KV
Act
Weights 4.0 GBKV-Cache 1.7 GBActivations 2.4 GBOverhead 0.3 GB

Precision Impact

bf16

4.0 GB

weights/GPU

~330.5 tok/s

fp8

2.0 GB

weights/GPU

int4

1.0 GB

weights/GPU

Quality Benchmarks

Bottom 25%
4th percentile across all models
MMLU
50.0
Bottom 25% (11th pctile)
HumanEval
22.0
Bottom 25% (4th pctile)
GSM8K
42.0
Bottom 25% (11th pctile)
MT-Bench
65.0
Bottom 25% (0th pctile)

Capabilities

Features

✗ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllmsglangtgiollama

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy Gemma 3 2B

Similar Models

Gemma 3 1B

1B params · dense

Quality: 35

Larger context, Lower qualityCompare →

Gemma 3 4B

4.3B params · dense

Quality: 54

from $0.10/M

Larger context, Higher quality, Larger modelCompare →

Moondream 2B

1.86B params · dense

Quality: 50

Smaller context, Higher qualityCompare →

Qwen 2 VL 2B

2.2B params · dense

Quality: 50

Larger context, Higher qualityCompare →

SeamlessM4T v2 Large

2.3B params · dense

Quality: 50

Higher qualityCompare →

Frequently Asked Questions

How much VRAM does Gemma 3 2B need for inference?

Gemma 3 2B requires approximately 4.0 GB of VRAM at BF16 precision, 2.0 GB at FP8, or 1.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (26624 bytes per token) and activations (~0.30 GB).

What is the best GPU for Gemma 3 2B?

The top recommended GPU for Gemma 3 2B is the RTX 4060 using BF16 precision. It achieves approximately 330.5 tokens/sec at an estimated cost of $209/month ($0.24/M tokens). Score: 100/100.

How much does Gemma 3 2B inference cost?

Gemma 3 2B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.