Leaderboard
Every model ranked on quality, throughput and real dollars per million tokens, with each figure traceable to its source.
Open the leaderboard317 models·60 GPUs·19 providers·Independently sourced, refreshed daily
Set GPUs, nodes and price, then read throughput and hourly cost before you commit a dollar. Try it right here.
Live rankings
Quality, throughput and real $/M tokens for every model, ranked, filterable and sourced.
Image, speech, embedding, video and scientific models, with their pricing basis.
| Model | Type | Provider | Arch | Pricing |
|---|---|---|---|---|
 Edify Image | image-gen |  NVIDIA | diffusion | $0.040 / image |
 Nemotron-3-Embed-1B-NVFP4 | image-gen | nvidia | encoder | Free · open weights |
| Nemotron-3-Embed-8B-BF16 | image-gen | nvidia | encoder | Free · open weights |
| StyleGAN3 | image-gen | NVIDIA | gan | Free · open weights |
| BioNeMo ESM-2 650M | protein | NVIDIA | encoder | Free · open weights |
| Riva FastPitch (en-US) | speech-tts | NVIDIA | encoder-decoder | Free · open weights |
| Riva HiFi-GAN (en-US) | speech-tts | NVIDIA | gan | Free · open weights |
| Maxine Eye Contact | video-fx | NVIDIA | pipeline | $0.000 / video min |
| DINOv2 ViT-g/14 (NVIDIA-optimized) | vision-embedding | NVIDIA | encoder | Free · open weights |
| NV-CLIP | vision-embedding | NVIDIA | encoder | Per embedding |
The standout model on each axis: best value, best quality, cheapest and fastest.
Every model plotted by quality against price, so the frontier and the outliers are obvious.
| Model | $/M | Q | |
|---|---|---|---|
| 1 | Gemini 2.0 FlashPick | $0.400 | 80 |
| 2 | DeepSeek V3 | $0.420 | 81 |
| 3 | HelpSteer2 Llama 3.1 70B | $0.500 | 82 |
| 4 | Nemotron 70B | $0.880 | 83 |
| 5 | Llama 3.2 90B Vision Instruct | $1.20 | 84 |
| 6 | DeepSeek R1 | $2.19 | 88 |
| 7 | Grok-3 | $15 | 91 |
| 8 | Llama 4 Behemoth | $16 | 93 |
The biggest provider price drops and the newest models, with the dates they landed.
Top model for Code, Math, Reasoning and Vision, plus each size class — no single model wins everything.
Chatbot, code generation, document analysis and more, each with a defensible starter model.
The top inference providers ranked by reputation, with model coverage and price ranges.
Reputation = pricing competitiveness · uptime · model coverage · feature parity
See all 10 providers →Market insight
From $0.005 to $150 per million output tokens across 189 priced models. The model you pick, not the task, drives the bill.
Compare on the leaderboardCheapest output price for each of 189 priced models, on a log scale. The model you choose can swing your bill by orders of magnitude.
How InferenceBench works
Run the open-source InferenceBench CLI to generate a signed record — the hardware, software stack, dataset hash and metrics, cryptographically sealed so anyone can verify the run.
Where the GPUs actually live: facilities, connectivity and carbon intensity across the global compute map.
Data centres by country
2,254 totalPlayground
Send one prompt to several models and compare the answers, latency and cost side by side.
We just shipped a public pricing API — live $/M tokens across 19 providers in one call. No key required. Full schema in the docs…
Streamed responses, side-by-side providers, real measured throughput. No login.
RAG is the right move when answers must cite source documents that change daily — search beats fine-tuning here.
Use RAG for fresh, citable knowledge; fine-tune when you need a fixed persona or domain-specific style.
Watch both stream at once. See latency, $/M tokens and answer quality side-by-side.
SELECT c.id, SUM(o.amount) AS rev FROM customers c JOIN orders o ON o.customer_id = c.id WHERE…
WITH last_q AS (SELECT * FROM orders WHERE quarter(date) = …) SELECT customer_id, SUM(amount)…
Two anonymous models answer. You pick the winner. Elo ranking updates live.
Build versus buy, the throughput bottleneck and a price forecast, each computed end to end.
Two cost paths for the same workload: rent the API per-token from the cheapest provider, or rent 2× H100 PCIe 24×7 and serve it yourself.
Decode is memory-bandwidth dominated at batch 1: every token reloads the full weight matrix. Compute sits idle. Splitting across 2 GPUs (TP=2) doubles the BW ceiling.
The spread between budget and premium $/M, today. Sparkline shows the sorted distribution on log scale.
Free, no signup — pick the part of the decision you care about.
Pick a model, set throughput, compare $/M tokens across 10 providers.
23 training methods, real GPU pricing, full epoch and GPU-hour breakdown.
Side by side: quality, latency, $/M tokens, context window, license.
Memory bandwidth, FP8 FLOPS, TDP, MSRP and current $/hr.
Pricing history, reliability score, regions and features per provider.
Describe what you are building, get a ranked model and GPU shortlist.
For developers
Everything behind the leaderboard as a free public API — no key required.
// Get the cheapest provider for Llama 3.1 70B
const res = await fetch(
'https://inferencebench.io/api/v1/models/meta-llama/llama-3.1-70b/pricing'
);
const { providers } = await res.json();
providers
.sort((a, b) => a.output_per_m - b.output_per_m)
.slice(0, 3)
.forEach((p) =>
console.log(`${p.provider} · $${p.output_per_m}/M`)
);
// DeepInfra · $0.40/M
// Groq · $0.79/M
// Openrouter · $0.79/MSame data the leaderboard renders from — exposed as plain JSON. No API key. Rate-limit generous.
/api/v1/models/api/v1/gpus/api/v1/providers/api/v1/pricing/api/v1/leaderboardFAQ
An AI inference benchmark measures how fast a GPU or cloud provider can generate tokens from a large language model (LLM). Key metrics include tokens per second (throughput), time to first token (TTFT), inter-token latency (ITL), and cost per million tokens.
InferenceBench uses a roofline performance model combined with CUDA kernel-level modeling (FlashAttention, PagedAttention, fused kernels) to predict real-world inference throughput. Results are validated against actual benchmarks from the HuggingFace LLM Perf Leaderboard and provider-reported data.
Performance depends on model size. For large models (70B+), the NVIDIA B200 and H200 lead in throughput. For mid-size models (7B–30B), the H100 SXM offers the best price-performance. For budget deployments, the RTX 4090 and L40S are strong contenders.
Pricing data is refreshed every 6 hours via automated API calls to providers. Benchmark results are updated when new GPU hardware or model architectures are released. Community-submitted data is verified before inclusion.