Updated minutes ago
Groq LPU
other · other · 230 GB SRAM · 300W TDP
VRAM
230 GB
BF16 TFLOPS
188
Bandwidth
80000 GB/s
From
$0.00/hr
Spec Sheet
VRAM230 GB SRAM
Memory Bandwidth80000 GB/s
BF16 TFLOPS188
FP16 TFLOPS188
FP8 TFLOPS376
INT8 TOPS750
TDP300W
InterconnectPCIE
Max per Node8
PCIe Gen5
Tensor CoresNo
Pricing by Provider
| Provider | On-Demand | Reserved | Spot | Badge |
|---|---|---|---|---|
| groq | $0.00/hr | - | - | Cheapest |
Compatible Models (251)
Single GPU (227 models)
Falcon 180B180B FP8Mixtral 8x22B141B FP8DBRX Base132B FP8DBRX Instruct132B FP8Mistral Large 2411123B FP8Mistral Large 2123B FP8Llama 4 Scout109B FP8Command R+104B FP8Yi-Large102.6B FP8YaLM 100B100B FP8Llama 3.2 90B Vision90B FP8Llama 3.2 90B Vision Instruct88.8B FP8Qwen 2.5 72B72.7B FP8Qwen 2.5 Math 72B72.7B FP8Qwen 2.5 VL 72B72.7B FP8Dolphin 2.9 72B72B FP8DeepSeek R1 Distill 70B70.6B FP8Llama 3 70B 1M Context70.6B FP8Llama 3 70B70.6B FP8Llama 3.1 70B70.6B FP8+207 more
Multi-GPU (24 models)
Nemotron 340Bx2 FP8Grok-2x2 FP8DeepSeek Coder V2 236Bx2 FP8DeepSeek V2.5x2 FP8Qwen 3 235Bx2 FP8Gemini 1.5 Prox2 BF16Claude 3 Opusx2 BF16Command Ax2 BF16Inflection 3x2 BF16Megatron-Turing NLG 530Bx3 FP8Snowflake Arctic 480Bx3 FP8Llama 3.1 405Bx3 FP8Llama 4 Maverickx3 FP8Jamba 1.5 Largex3 FP8Snowflake Arctic 128x3Bx3 FP8+9 more
Training Capabilities
Estimated GPU count for full fine-tuning (AdamW, BF16) and QLoRA
| Model Size | Full Fine-Tune | QLoRA |
|---|---|---|
| 7B model | 1 GPU | 1 GPU |
| 13B model | 2 GPUs | 1 GPU |
| 70B model | 6 GPUs | 1 GPU |
Energy Efficiency
Estimated tokens/second per Watt for popular models
Mistral 7B
36.53 t/s/WFP8
Qwen 2.5 7B
35.09 t/s/WFP8
Llama 3.1 8B
33.21 t/s/WFP8
DeepSeek V3
7.21 t/s/WFP8
Llama 3.1 70B
3.78 t/s/WFP8
Qwen 2.5 72B
3.67 t/s/WFP8
Similar GPUs
| GPU | VRAM | BF16 TFLOPS | BW (GB/s) | From |
|---|---|---|---|---|
| Trainium2 | 96 GB | 756 | 3200 | $1.95/hr |
| Cloud AI 100 | 32 GB | 150 | 134 | $0.00/hr |
| Instinct MI325X | 256 GB | 1307 | 6000 | $2.49/hr |
| B100 SXM | 192 GB | 1750 | 8000 | $4.50/hr |
| GB200 NVL72 (per GPU) | 192 GB | 2250 | 8000 | $6.50/hr |