GPU × use-case guide · Speech-to-text

Is the H100 NVL 94GB (per GPU pair) a good GPU for speech-to-text?

The H100 NVL 94GB (per GPU pair) is a hopper NVIDIA GPU with 188 GB HBM3 (7876 GB/s), 1670 TFLOPS BF16 / 3341 TFLOPS FP8, and a 800 W TDP. Speech-to-text workloads care most about encoder throughput and real-time factor; models are small so mid-range cards are plenty. Here's how the H100 NVL 94GB (per GPU pair) measures up.

What models fit on a single H100 NVL 94GB (per GPU pair)?

Weights only, reserving ~25% of the 188 GB for KV cache, activations and fragmentation. ✓ = fits on one card.

Largest single-card fit: Llama 3.3 70B at BF16, Qwen 2.5 72B at FP8, Qwen 2.5 72B at INT4. Bigger models need tensor-parallel across 4 cards.

H100 NVL 94GB (per GPU pair) for speech-to-text, specifically

Speech-to-text is throughput-bound rather than VRAM-bound, so the H100 NVL 94GB (per GPU pair)'s 7876 GB/s of bandwidth and 1670 TFLOPS matter more than raw capacity. Because it's latency-sensitive, run it at low batch sizes on a fast framework (vLLM ≈ 85% MFU) rather than maximising batch. Size it precisely on the calculator.

H100 NVL 94GB (per GPU pair) pricing across providers

Verdict

With 188 GB, the H100 NVL 94GB (per GPU pair) is a data-center-class card that comfortably handles speech-to-text for models up to Llama 3.3 70B at full precision on a single card — a strong pick if your budget supports ~$7.49/hr.