Updated minutes ago· Sources: GPU Pricing, API Token Pricing, Model Registry

A16

nvidia · ampere · 64 GB GDDR6 · 250W TDP

VRAM

64 GB

BF16 TFLOPS

16.8

Bandwidth

232 GB/s

From

$0.72/hr

Calculate ROI with this GPU →

Spec Sheet

VRAM64 GB GDDR6

Memory Bandwidth232 GB/s

BF16 TFLOPS16.8

FP16 TFLOPS16.8

FP8 TFLOPS16.8

INT8 TOPS33.6

TDP250W

InterconnectPCIE

Max per Node8

PCIe Gen4

CUDA Compute Capability8.6

Tensor CoresYes

Pricing by Provider

Provider	On-Demand	Reserved	Spot	Badge
aws	$1.10/hr	$0.72/hr	-	Cheapest

Compatible Models (248)

Multi-GPU (63 models)

Command R+x2 FP8 Yi-Largex2 FP8 YaLM 100Bx2 FP8 Llama 3.2 90B Visionx2 FP8 Llama 3.2 90B Vision Instructx2 FP8 Qwen 2.5 72Bx2 FP8 Qwen 2.5 Math 72Bx2 FP8 Qwen 2.5 VL 72Bx2 FP8 Dolphin 2.9 72Bx2 FP8 DeepSeek R1 Distill 70Bx2 FP8 Llama 3 70B 1M Contextx2 FP8 Llama 3 70Bx2 FP8 Llama 3.1 70Bx2 FP8 Llama 3.3 70Bx2 FP8 Hermes 3 70Bx2 FP8+48 more

Training Capabilities

Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA

Model Size	Full Fine-Tune	QLoRA
7B model	3 GPUs	1 GPU
13B model	4 GPUs	1 GPU
70B model	21 GPUs	1 GPU

Train on this GPU →

Energy Efficiency

Estimated tokens/second per Watt for popular models

Mistral 7B

0.13 t/s/WFP8

Qwen 2.5 7B

0.12 t/s/WFP8

Llama 3.1 8B

0.12 t/s/WFP8

Llama 3.1 70B

0.01 t/s/WFP8

Qwen 2.5 72B