Updated minutes ago
H100 PCIe
nvidia · hopper · 80 GB HBM3 · 350W TDP
VRAM
80 GB
BF16 TFLOPS
756
Bandwidth
2000 GB/s
From
$1.89/hr
Spec Sheet
VRAM80 GB HBM3
Memory Bandwidth2000 GB/s
BF16 TFLOPS756
FP16 TFLOPS756
FP8 TFLOPS1513
INT8 TOPS1513
TDP350W
InterconnectPCIE
Max per Node8
PCIe Gen5
CUDA Compute Capability9
Tensor CoresYes
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| tensordock | $2.59/hr | - | $1.89/hr | Cheapest |
| vast_ai | $2.80/hr | - | $2.10/hr | |
| lambda | $2.29/hr | - | - | |
| runpod | $3.09/hr | - | $2.39/hr |
Compatible Models (249)
Single GPU (186 models)
DeepSeek LLM 67B67B FP8Jamba 1.5 Mini52B FP8Llama 3.1 Nemotron 51B51B FP8Amazon Nova Pro50B FP8Mixtral 8x7B46.7B FP8Mixtral 8x7B Instruct46.7B FP8Phi 3.5 MoE41.9B FP8Falcon 40B40B FP8VILA 1.5 40B40B FP8Aya 23 35B35B FP8Command R35B FP8Command R (August 2024)35B FP8Yi 1.5 34B34.4B FP8Code Llama 34B34B FP8DeepSeek Coder 33B33B FP8Vicuna 33B33B FP8WizardCoder 33B33B FP8DeepSeek R1 Distill 32B32.8B FP8Qwen 3 32B32.8B FP8Qwen 2.5 32B32.5B FP8+166 more
Multi-GPU (63 models)
DBRX Basex2 FP8DBRX Instructx2 FP8Mistral Large 2411x2 FP8Mistral Large 2x2 FP8Llama 4 Scoutx2 FP8Command R+x2 FP8Yi-Largex2 FP8YaLM 100Bx2 FP8Llama 3.2 90B Visionx2 FP8Llama 3.2 90B Vision Instructx2 FP8Qwen 2.5 72Bx2 FP8Qwen 2.5 Math 72Bx2 FP8Qwen 2.5 VL 72Bx2 FP8Dolphin 2.9 72Bx2 FP8DeepSeek R1 Distill 70Bx2 FP8+48 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 2 GPUs | 1 GPU |
| 13B model | 4 GPUs | 1 GPU |
| 70B model | 17 GPUs | 1 GPU |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.78 t/s/WFP8
Qwen 2.5 7B
0.75 t/s/WFP8
Llama 3.1 8B
0.71 t/s/WFP8
Llama 3.1 70B
0.08 t/s/WFP8
Qwen 2.5 72B
0.08 t/s/WFP8