GPU × use-case guide · Vision-language

Is the Instinct MI300X a good GPU for vision-language?

The Instinct MI300X is a cdna3 AMD GPU with 192 GB HBM3 (5300 GB/s), 1307 TFLOPS BF16 / 2614 TFLOPS FP8, and a 750 W TDP. Vision-language workloads care most about extra VRAM for the vision encoder and image tokens on top of the language model. Here's how the Instinct MI300X measures up.

What models fit on a single Instinct MI300X?

Weights only, reserving ~25% of the 192 GB for KV cache, activations and fragmentation. ✓ = fits on one card.

Largest single-card fit: Qwen 2.5 72B at BF16, Qwen 2.5 72B at FP8, Qwen 2.5 72B at INT4. Bigger models need tensor-parallel across 8 cards.

Instinct MI300X for vision-language, specifically

Vision-language is context-heavy, so the KV cache — not the weights — is what fills the 192 GB. On the Instinct MI300X you'll trade context length against batch size: long prompts mean fewer concurrent requests. Because it runs offline, batch aggressively to push tokens-per-dollar down. Size it precisely on the calculator.

Instinct MI300X pricing across providers

Verdict

With 192 GB, the Instinct MI300X is a data-center-class card that comfortably handles vision-language for models up to Qwen 2.5 72B at full precision on a single card — a strong pick if your budget supports ~$2.39/hr.