Updated minutes ago
Amazon

Amazon Nova Pro

Amazon · dense · 50B parameters · 300,000 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

50B

Context Window

293K tokens

Architecture

Dense

Best GPU

H100 SXM

Cheapest API

$3.20/M

Intelligence Brief

Amazon Nova Pro is a 50B parameter DENSE model from Amazon, featuring Grouped Query Attention (GQA) with 60 layers and 8,192 hidden dimensions. With a 300,000 token context window, it supports tools, vision, structured output, code, math, multilingual. The most cost-effective API deployment is via amazon at $3.20/M output tokens. For self-hosted inference, H100 SXM delivers optimal throughput at $1794/month.

Architecture Details

TypeDENSE
Total Parameters50B
Active Parameters50B
Layers60
Hidden Dimension8,192
Attention Heads64
KV Heads8
Head Dimension128
Vocab Size128,000

Memory Requirements

BF16 Weights

100.0 GB

FP8 Weights

50.0 GB

INT4 Weights

25.0 GB

KV-Cache per Token245760 bytes
Activation Estimate2.00 GB

Fits on (single GPU) — most practical first

BEST FITRTX 509032GBINT4V100 32GB32GBINT4Instinct MI10032GBINT4TPU v432GBINT4TPU v6e (Trillium)32GBINT4Cloud AI 10032GBINT4A100 40GB SXM40GBINT4A100 40GB PCIe40GBINT4RTX A600048GBINT4A4048GBINT4L40S48GBINT4L4048GBINT4

GPU Compatibility Matrix

Amazon Nova Pro is compatible with 40% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
72%
86%
93%
B300288GB
65%
83%
91%
B100 SXM192GB
48%
74%
87%
GB200 NVL72 (per GPU)192GB
48%
74%
87%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
47%
73%
87%
H200 SXM141GB
29%
65%
82%
H2096GB
✗
48%
74%
GH20096GB
✗
48%
74%
Ada Lovelace(11 GPUs)
L40S48GB
✗
✗
48%
L4048GB
✗
✗
48%
RTX 6000 Ada48GB
✗
✗
48%
L2048GB
✗
✗
48%
Ampere(16 GPUs)
A100 80GB SXM80GB
✗
38%
69%
A100 80GB PCIe80GB
✗
38%
69%
A1664GB
✗
22%
61%
RTX A600048GB
✗
✗
48%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$1.22

Use this config →
H100 PCIeoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

334.6 tok/s

Latency (ITL)

3.0ms

Est. TTFT

1ms

Cost/Month

$1794

Cost/M Tokens

$2.04

Use this config →
H100 NVLoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

560.0 tok/s

Latency (ITL)

1.8ms

Est. TTFT

0ms

Cost/Month

$2932

Cost/M Tokens

$1.99

Use this config →

Deployment Options

API

API Deployment

amazon

$3.20/M

output tokens

Self-Hosted

Single GPU

H100 SXM

$1794/mo

Min VRAM: 50 GB

Scale

Multi-GPU

A100 80GB SXM x2

217.6 tok/s

TP· $2259/mo

Open Full Calculator →

API Pricing Comparison

ProviderInput $/MOutput $/MBadges
amazon$0.80$3.20
Cheapest

Cost Analysis

ProviderInput $/MOutput $/M~Monthly Cost
amazonBest Value$0.80$3.20$20

Cost per 1,000 Requests

Short (500 tok)

$1.04

via amazon

Medium (2K tok)

$4.16

via amazon

Long (8K tok)

$12.80

via amazon

Performance Estimates

Throughput by GPU

H100 SXM
560.0 tok/s
H100 PCIe
334.6 tok/s
H100 NVL
560.0 tok/s

VRAM Breakdown (H100 SXM, FP8)

Weights
Act
Weights 50.0 GBKV-Cache 2.0 GBActivations 16.0 GBOverhead 2.5 GB

Precision Impact

bf16

100.0 GB

weights/GPU

fp8

50.0 GB

weights/GPU

~560.0 tok/s

Capabilities

Features

✓ Tool Use✓ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllm

Supported Precisions

BF16 (default)FP8

Where to Deploy Amazon Nova Pro

Similar Models

Gemini 2.0 Flash

50B params · moe

Quality: 80

from $0.40/M

Larger context, Higher quality, CheaperCompare →

Gemini 1.5 Flash

50B params · moe

Quality: 75

from $0.30/M

Larger context, Higher quality, CheaperCompare →

Llama 3.1 Nemotron 51B

51B params · dense

Quality: 78

from $0.40/M

Smaller context, Higher quality, CheaperCompare →

Jamba 1.5 Mini

52B params · hybrid

Quality: 50

from $0.40/M

CheaperCompare →

Jamba Instruct

52B params · moe

Quality: 66

from $0.70/M

Higher quality, CheaperCompare →

Frequently Asked Questions

How much VRAM does Amazon Nova Pro need for inference?

Amazon Nova Pro requires approximately 100.0 GB of VRAM at BF16 precision, 50.0 GB at FP8, or 25.0 GB at INT4 quantization. Additional VRAM is needed for KV-cache (245760 bytes per token) and activations (~2.00 GB).

What is the best GPU for Amazon Nova Pro?

The top recommended GPU for Amazon Nova Pro is the H100 SXM using FP8 precision. It achieves approximately 560.0 tokens/sec at an estimated cost of $1794/month ($1.22/M tokens). Score: 100/100.

How much does Amazon Nova Pro inference cost?

Amazon Nova Pro API inference starts from $0.80/M input tokens and $3.20/M output tokens. Self-hosted inference costs depend on your GPU configuration — use our ROI calculator for a detailed breakdown.