GPU × use-case guide · Vision-language

Is the H200 SXM a good GPU for vision-language?

The H200 SXM is a hopper NVIDIA GPU with 141 GB HBM3e (4800 GB/s), 990 TFLOPS BF16 / 1979 TFLOPS FP8, and a 700 W TDP. Vision-language workloads care most about extra VRAM for the vision encoder and image tokens on top of the language model. Here's how the H200 SXM measures up.

What models fit on a single H200 SXM?

Weights only, reserving ~25% of the 141 GB for KV cache, activations and fragmentation. ✓ = fits on one card.

Largest single-card fit: Mixtral 8x7B (MoE) at BF16, Qwen 2.5 72B at FP8, Qwen 2.5 72B at INT4. Bigger models need tensor-parallel across 8 cards.

H200 SXM for vision-language, specifically

Vision-language is context-heavy, so the KV cache — not the weights — is what fills the 141 GB. On the H200 SXM you'll trade context length against batch size: long prompts mean fewer concurrent requests. Because it runs offline, batch aggressively to push tokens-per-dollar down. Size it precisely on the calculator.

H200 SXM pricing across providers

Verdict

With 141 GB, the H200 SXM is a data-center-class card that comfortably handles vision-language for models up to Mixtral 8x7B (MoE) at full precision on a single card — a strong pick if your budget supports ~$3.49/hr.