Updated minutes ago
RTX 6000 Ada
nvidia · ada · 48 GB GDDR6 · 300W TDP
VRAM
48 GB
BF16 TFLOPS
91.1
Bandwidth
960 GB/s
From
$0.59/hr
Spec Sheet
VRAM48 GB GDDR6
Memory Bandwidth960 GB/s
BF16 TFLOPS91.1
FP16 TFLOPS91.1
FP8 TFLOPS182.2
INT8 TOPS182.2
TDP300W
InterconnectPCIE
Max per Node8
PCIe Gen4
CUDA Compute Capability8.9
Tensor CoresYes
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| tensordock | $0.79/hr | - | $0.59/hr | Cheapest |
| vast_ai | $0.89/hr | - | $0.65/hr | |
| runpod | $1.29/hr | - | $0.99/hr | |
| lambda | $1.09/hr | - | - |
Compatible Models (239)
Single GPU (179 models)
Falcon 40B40B FP8VILA 1.5 40B40B FP8Aya 23 35B35B FP8Command R35B FP8Command R (August 2024)35B FP8Yi 1.5 34B34.4B FP8Code Llama 34B34B FP8DeepSeek Coder 33B33B FP8Vicuna 33B33B FP8WizardCoder 33B33B FP8DeepSeek R1 Distill 32B32.8B FP8Qwen 3 32B32.8B FP8Qwen 2.5 32B32.5B FP8Qwen 2.5 Coder 32B32.5B FP8Qwen 3 30B-A3B30.5B FP8JAIS 30B30B FP8MPT 30B30B FP8Gemma 2 27B27B FP8Gemma 3 27B27B FP8InternVL2 26B26B FP8+159 more
Multi-GPU (60 models)
Qwen 2.5 72Bx2 FP8Qwen 2.5 Math 72Bx2 FP8Qwen 2.5 VL 72Bx2 FP8Dolphin 2.9 72Bx2 FP8DeepSeek R1 Distill 70Bx2 FP8Llama 3 70B 1M Contextx2 FP8Llama 3 70Bx2 FP8Llama 3.1 70Bx2 FP8Llama 3.3 70Bx2 FP8Hermes 3 70Bx2 FP8HelpSteer2 Llama 3.1 70Bx2 FP8Llama 3.1 Nemotron 70B Instructx2 FP8Llama 3.1 Nemotron 70B Rewardx2 FP8Nemotron 70Bx2 FP8Llama 3.1 70B Turbox2 FP8+45 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 3 GPUs | 1 GPU |
| 13B model | 6 GPUs | 1 GPU |
| 70B model | 28 GPUs | 1 GPU |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.44 t/s/WFP8
Qwen 2.5 7B
0.42 t/s/WFP8
Llama 3.1 8B
0.40 t/s/WFP8
Llama 3.1 70B
0.05 t/s/WFP8
Qwen 2.5 72B
0.04 t/s/WFP8