Updated minutes ago
Trainium2
other · other · 96 GB HBM2e · 600W TDP
VRAM
96 GB
BF16 TFLOPS
756
Bandwidth
3200 GB/s
From
$1.95/hr
Spec Sheet
VRAM96 GB HBM2e
Memory Bandwidth3200 GB/s
BF16 TFLOPS756
FP16 TFLOPS756
FP8 TFLOPS1512
INT8 TOPS1512
TDP600W
InterconnectPCIE
Max per Node16
PCIe Gen5
Tensor CoresNo
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| aws | $2.78/hr | $1.95/hr | - | Cheapest |
Compatible Models (251)
Single GPU (207 models)
Qwen 2.5 72B72.7B FP8Qwen 2.5 Math 72B72.7B FP8Qwen 2.5 VL 72B72.7B FP8Dolphin 2.9 72B72B FP8DeepSeek R1 Distill 70B70.6B FP8Llama 3 70B 1M Context70.6B FP8Llama 3 70B70.6B FP8Llama 3.1 70B70.6B FP8Llama 3.3 70B70.6B FP8Hermes 3 70B70.6B FP8HelpSteer2 Llama 3.1 70B70.6B FP8Llama 3.1 Nemotron 70B Instruct70.6B FP8Llama 3.1 Nemotron 70B Reward70.6B FP8Nemotron 70B70.6B FP8Llama 3.1 70B Turbo70.6B FP8Code Llama 70B70B FP8Llama 2 70B70B FP8WizardMath 70B70B FP8Meditron 70B70B FP8Japanese StableLM 70B70B FP8+187 more
Multi-GPU (44 models)
Mixtral 8x22Bx2 FP8DBRX Basex2 FP8DBRX Instructx2 FP8Mistral Large 2411x2 FP8Mistral Large 2x2 FP8Llama 4 Scoutx2 FP8Command R+x2 FP8Yi-Largex2 FP8YaLM 100Bx2 FP8Llama 3.2 90B Visionx2 FP8Llama 3.2 90B Vision Instructx2 FP8Claude Sonnet 4x2 BF16o1-minix2 BF16o3-minix2 BF16Claude 3 Sonnetx2 BF16+29 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 2 GPUs | 1 GPU |
| 13B model | 3 GPUs | 1 GPU |
| 70B model | 14 GPUs | 1 GPU |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.73 t/s/WFP8
Qwen 2.5 7B
0.70 t/s/WFP8
Llama 3.1 8B
0.66 t/s/WFP8
DeepSeek V3
0.14 t/s/WFP8
Llama 3.1 70B
0.08 t/s/WFP8
Qwen 2.5 72B
0.07 t/s/WFP8
Similar GPUs
| GPU | VRAM | BF16 TFLOPS | BW (GB/s) | From |
|---|---|---|---|---|
| Cloud AI 100 | 32 GB | 150 | 134 | $0.00/hr |
| Groq LPU | 230 GB | 188 | 80000 | $0.00/hr |
| H20 | 96 GB | 148 | 4000 | $0.99/hr |
| Gaudi 2 | 96 GB | 432 | 2460 | $1.65/hr |
| GH200 | 96 GB | 990 | 4000 | $2.99/hr |