GPU × use-case guide · Vision-language

Is the RTX 4090 a good GPU for vision-language?

The RTX 4090 is a ada NVIDIA GPU with 24 GB GDDR6X (1008 GB/s), 165 TFLOPS BF16 / 330 TFLOPS FP8, and a 450 W TDP. Vision-language workloads care most about extra VRAM for the vision encoder and image tokens on top of the language model. Here's how the RTX 4090 measures up.

What models fit on a single RTX 4090?

Weights only, reserving ~25% of the 24 GB for KV cache, activations and fragmentation. ✓ = fits on one card.

Largest single-card fit: Llama 3.1 8B at BF16, Qwen 2.5 14B at FP8, Gemma 2 27B at INT4. Bigger models need tensor-parallel across 4 cards.

RTX 4090 for vision-language, specifically

Vision-language is context-heavy, so the KV cache — not the weights — is what fills the 24 GB. On the RTX 4090 you'll trade context length against batch size: long prompts mean fewer concurrent requests. Because it runs offline, batch aggressively to push tokens-per-dollar down. Size it precisely on the calculator.

RTX 4090 pricing across providers

Verdict

At 24 GB, the RTX 4090 is a value card best suited to smaller vision-language models (up to Gemma 2 27B with INT4 quantisation). For larger models you'll want more VRAM or multi-GPU.