Updated minutes ago
A100 80GB PCIe
nvidia · ampere · 80 GB HBM2e · 300W TDP
VRAM
80 GB
BF16 TFLOPS
312
Bandwidth
2039 GB/s
From
$1.05/hr
Spec Sheet
VRAM80 GB HBM2e
Memory Bandwidth2039 GB/s
BF16 TFLOPS312
FP16 TFLOPS312
FP8 TFLOPS312
INT8 TOPS624
TDP300W
InterconnectPCIE
Max per Node8
PCIe Gen4
CUDA Compute Capability8
Tensor CoresYes
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| fluidstack | $1.49/hr | - | $1.05/hr | Cheapest |
| tensordock | $1.59/hr | - | $1.09/hr | |
| vast_ai | $1.60/hr | - | $1.10/hr | |
| runpod | $2.29/hr | - | $1.79/hr | |
| lambda | $1.79/hr | - | - |
Compatible Models (249)
Single GPU (186 models)
DeepSeek LLM 67B67B FP8Jamba 1.5 Mini52B FP8Llama 3.1 Nemotron 51B51B FP8Amazon Nova Pro50B FP8Mixtral 8x7B46.7B FP8Mixtral 8x7B Instruct46.7B FP8Phi 3.5 MoE41.9B FP8Falcon 40B40B FP8VILA 1.5 40B40B FP8Aya 23 35B35B FP8Command R35B FP8Command R (August 2024)35B FP8Yi 1.5 34B34.4B FP8Code Llama 34B34B FP8DeepSeek Coder 33B33B FP8Vicuna 33B33B FP8WizardCoder 33B33B FP8DeepSeek R1 Distill 32B32.8B FP8Qwen 3 32B32.8B FP8Qwen 2.5 32B32.5B FP8+166 more
Multi-GPU (63 models)
DBRX Basex2 FP8DBRX Instructx2 FP8Mistral Large 2411x2 FP8Mistral Large 2x2 FP8Llama 4 Scoutx2 FP8Command R+x2 FP8Yi-Largex2 FP8YaLM 100Bx2 FP8Llama 3.2 90B Visionx2 FP8Llama 3.2 90B Vision Instructx2 FP8Qwen 2.5 72Bx2 FP8Qwen 2.5 Math 72Bx2 FP8Qwen 2.5 VL 72Bx2 FP8Dolphin 2.9 72Bx2 FP8DeepSeek R1 Distill 70Bx2 FP8+48 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 2 GPUs | 1 GPU |
| 13B model | 4 GPUs | 1 GPU |
| 70B model | 17 GPUs | 1 GPU |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.93 t/s/WFP8
Qwen 2.5 7B
0.89 t/s/WFP8
Llama 3.1 8B
0.85 t/s/WFP8
Llama 3.1 70B
0.10 t/s/WFP8
Qwen 2.5 72B
0.09 t/s/WFP8
Similar GPUs
| GPU | VRAM | BF16 TFLOPS | BW (GB/s) | From |
|---|---|---|---|---|
| A100 80GB SXM | 80 GB | 312 | 2039 | $1.19/hr |
| A16 | 64 GB | 16.8 | 232 | $0.72/hr |
| RTX A6000 | 48 GB | 38.7 | 768 | $0.49/hr |
| A40 | 48 GB | 37.4 | 696 | $0.42/hr |
| A100 40GB SXM | 40 GB | 312 | 1555 | $0.85/hr |