Updated minutes ago
A100 40GB PCIe
nvidia · ampere · 40 GB HBM2e · 250W TDP
VRAM
40 GB
BF16 TFLOPS
312
Bandwidth
1555 GB/s
From
$0.69/hr
Spec Sheet
VRAM40 GB HBM2e
Memory Bandwidth1555 GB/s
BF16 TFLOPS312
FP16 TFLOPS312
FP8 TFLOPS312
INT8 TOPS624
TDP250W
InterconnectPCIE
Max per Node8
PCIe Gen4
CUDA Compute Capability8
Tensor CoresYes
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| tensordock | $0.99/hr | - | $0.69/hr | Cheapest |
| vast_ai | $1.10/hr | - | $0.75/hr | |
| runpod | $1.44/hr | - | $1.09/hr | |
| lambda | $1.10/hr | - | - |
Compatible Models (238)
Single GPU (171 models)
Code Llama 34B34B FP8DeepSeek Coder 33B33B FP8Vicuna 33B33B FP8WizardCoder 33B33B FP8DeepSeek R1 Distill 32B32.8B FP8Qwen 3 32B32.8B FP8Qwen 2.5 32B32.5B FP8Qwen 2.5 Coder 32B32.5B FP8Qwen 3 30B-A3B30.5B FP8JAIS 30B30B FP8MPT 30B30B FP8Gemma 2 27B27B FP8Gemma 3 27B27B FP8InternVL2 26B26B FP8Mistral Small 24B24B FP8Mistral Small 3.1 24B24B FP8Codestral 22B22B FP8Solar Pro 22B22B FP8GigaChat 20B20B FP8InternLM 20B20B FP8+151 more
Multi-GPU (67 models)
DeepSeek LLM 67Bx2 FP8Jamba 1.5 Minix2 FP8Llama 3.1 Nemotron 51Bx2 FP8Amazon Nova Prox2 FP8Mixtral 8x7Bx2 FP8Mixtral 8x7B Instructx2 FP8Phi 3.5 MoEx2 FP8Falcon 40Bx2 FP8VILA 1.5 40Bx2 FP8Aya 23 35Bx2 FP8Command Rx2 FP8Command R (August 2024)x2 FP8Yi 1.5 34Bx2 FP8Claude 3.5 Haikux2 BF16GPT-3.5 Turbox2 BF16+52 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 4 GPUs | 1 GPU |
| 13B model | 7 GPUs | 1 GPU |
| 70B model | 33 GPUs | 2 GPUs |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.85 t/s/WFP8
Qwen 2.5 7B
0.82 t/s/WFP8
Llama 3.1 8B
0.77 t/s/WFP8
Llama 3.1 70B
0.09 t/s/WFP8
Qwen 2.5 72B
0.09 t/s/WFP8
Similar GPUs
| GPU | VRAM | BF16 TFLOPS | BW (GB/s) | From |
|---|---|---|---|---|
| A100 40GB SXM | 40 GB | 312 | 1555 | $0.85/hr |
| RTX A6000 | 48 GB | 38.7 | 768 | $0.49/hr |
| A40 | 48 GB | 37.4 | 696 | $0.42/hr |
| A10G | 24 GB | 35 | 600 | $0.30/hr |
| A30 | 24 GB | 165 | 933 | $0.35/hr |