Updated minutes ago
Google

Gemini 1.5 Flash

Google · moe · 50B parameters · 1,048,576 context

Quality
75.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

50B

Context Window

1024K tokens

Architecture

MoE

Best GPU

B200 SXM

Cheapest API

$0.30/M

Quality Score

75/100

Intelligence Brief

Gemini 1.5 Flash is a 50B parameter Mixture-of-Experts (8 experts, 2 active) model from Google, featuring Grouped Query Attention (GQA) with 40 layers and 5,120 hidden dimensions. With a 1,048,576 token context window, it supports tools, vision, structured output, code, math, multilingual. On standardized benchmarks, it achieves MMLU 78.9, HumanEval 52, GSM8K 82. The most cost-effective API deployment is via google at $0.30/M output tokens. For self-hosted inference, B200 SXM delivers optimal throughput at $4261/month.

Architecture Details

TypeMOE
Total Parameters50B
Active Parameters12B
Layers40
Hidden Dimension5,120
Attention Heads40
KV Heads8
Head Dimension128
Vocab Size256,000
Total Experts8
Active Experts2

Memory Requirements

BF16 Weights

100.0 GB

FP8 Weights

50.0 GB

INT4 Weights

25.0 GB

KV-Cache per Token81920 bytes
Activation Estimate2.00 GB

Fits on (single GPU) — most practical first

BEST FITRTX 509032GBINT4V100 32GB32GBINT4Instinct MI10032GBINT4TPU v432GBINT4TPU v6e (Trillium)32GBINT4Cloud AI 10032GBINT4A100 40GB SXM40GBINT4A100 40GB PCIe40GBINT4RTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4

GPU Compatibility Matrix

Gemini 1.5 Flash is compatible with 40% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
72%
86%
93%
B300288GB
65%
83%
91%
B100 SXM192GB
48%
74%
87%
GB200 NVL72 (per GPU)192GB
48%
74%
87%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
47%
73%
87%
H200 SXM141GB
29%
65%
82%
H2096GB
✗
48%
74%
GH20096GB
✗
48%
74%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
48%
L4048GB
✗
✗
48%
RTX 6000 Ada48GB
✗
✗
48%
L2048GB
✗
✗
48%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
38%
69%
A100 80GB PCIe80GB
✗
38%
69%
A1664GB
✗
22%
61%
RTX A600048GB
✗
✗
48%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

B200 SXMoptimal

BF16 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$4261

Cost/M Tokens

$2.90

Use this config →
B100 SXMoptimal

BF16 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$4271

Cost/M Tokens

$2.90

Use this config →
GB200 NVL72 (per GPU)optimal

BF16 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$6169

Cost/M Tokens

$4.19

Use this config →

Deployment Options

API

API Deployment

google

$0.30/M

output tokens

Self-Hosted

Single GPU

B200 SXM

$4261/mo

Min VRAM: 50 GB

Scale

Multi-GPU

H20 x2

560.0 tok/s

TP· $1879/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
google$0.07$0.30
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
googleBest Value$0.07$0.30$2

Cost per 1,000 Requests

Short (500 tok)

$0.10

via google

Medium (2K tok)

$0.39

via google

Long (8K tok)

$1.20

via google

Performance Estimates

Throughput by GPU

B200 SXM
560.0 tok/s
B100 SXM
560.0 tok/s
GB200 NVL72 (per GPU)
560.0 tok/s

VRAM Breakdown (B200 SXM, BF16)

Weights
Weights 100.0 GBKV-Cache 2.7 GBActivations 16.0 GBOverhead 5.0 GB

Quality Benchmarks

Above Average
80th percentile across all models
MMLU
78.9
Average (55th pctile)
HumanEval
52.0
Average (50th pctile)
GSM8K
82.0
Average (50th pctile)
MT-Bench
80.0
Bottom 25% (0th pctile)

Capabilities

Features

✓ Tool Use✓ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

Supported Precisions

BF16 (default)

Where to Deploy Gemini 1.5 Flash

Similar Models

Gemini 2.0 Flash

50B params · moe

Quality: 80

from $0.40/M

Similar specsCompare →

Amazon Nova Pro

50B params · dense

Quality: 50

from $3.20/M

Smaller context, Lower quality, More expensiveCompare →

Llama 3.1 Nemotron 51B

51B params · dense

Quality: 78

from $0.40/M

Smaller contextCompare →

Jamba 1.5 Mini

52B params · hybrid

Quality: 50

from $0.40/M

Smaller context, Lower qualityCompare →

Jamba Instruct

52B params · moe

Quality: 66

from $0.70/M

Smaller context, Lower quality, More expensiveCompare →

Frequently Asked Questions

How much VRAM does Gemini 1.5 Flash need for inference?

Gemini 1.5 Flash requires approximately 100.0 GB of VRAM at BF16 precision, 50.0 GB at FP8, or 25.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (81920 bytes per token) and activations (~2.00 GB).

What is the best GPU for Gemini 1.5 Flash?

The top recommended GPU for Gemini 1.5 Flash is the B200 SXM using BF16 precision. It achieves approximately 560.0 tokens/sec at an estimated cost of $4261/month ($2.90/M tokens). Score: 100/100.

How much does Gemini 1.5 Flash inference cost?

Gemini 1.5 Flash API inference starts from $0.07/M input tokens and $0.30/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.