Updated minutes ago
RTX 4060
nvidia · ada · 8 GB GDDR6 · 115W TDP
VRAM
8 GB
BF16 TFLOPS
30
Bandwidth
272 GB/s
From
$0.22/hr
Spec Sheet
VRAM8 GB GDDR6
Memory Bandwidth272 GB/s
BF16 TFLOPS30
FP16 TFLOPS30
FP8 TFLOPS60
INT8 TOPS60
TDP115W
InterconnectPCIE
Max per Node1
PCIe Gen4
CUDA Compute Capability8.9
Tensor CoresYes
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| retail | $0.22/hr | - | - | Cheapest |
Compatible Models (131)
Single GPU (131 models)
OLMo 2 13B13B INT4Baichuan 2 13B13B INT4Vicuna 13B13B INT4Code Llama 13B13B INT4Llama 2 13B13B INT4Orca 2 13B13B INT4VILA 1.5 13B13B INT4ELYZA 13B13B INT4Cerebras GPT 13B13B INT4KULLM 12.8B12.8B INT4StableLM 2 12B12.1B INT4Gemma 3 12B12B INT4Mistral Nemo 12B12B INT4Pixtral 12B12B INT4Llama 3.2 11B Vision11B INT4Falcon 11B11B INT4SOLAR 10.7B10.7B INT4GLM-4 9B9.4B INT4ChatGLM4 9B9.4B INT4Gemma 2 9B9.2B INT4+111 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 17 GPUs | 1 GPU |
| 13B model | 31 GPUs | 1 GPU |
| 70B model | 165 GPUs | 6 GPUs |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
0.32 t/s/WFP8
Similar GPUs
| GPU | VRAM | BF16 TFLOPS | BW (GB/s) | From |
|---|---|---|---|---|
| RTX 4070 Ti | 12 GB | 93 | 504 | $0.25/hr |
| RTX 4070 Super | 12 GB | 55 | 504 | $0.22/hr |
| RTX 4080 | 16 GB | 97 | 717 | $0.32/hr |
| RTX 4060 Ti 16GB | 16 GB | 44 | 288 | $0.30/hr |
| L4 | 24 GB | 121 | 300 | $0.29/hr |