Signed benchmark leaderboard
52 cryptographically signed, hardware-fingerprinted benchmark envelopes across 11 suites. Every row ties to a real run you can re-verify with bench verify. Sourced from the public corpus on Hugging Face.
| Model | Engine | Hardware | pass@1 | TTFT p50 | Latency p50 | Tokens out | Timeout | OK rate | Signed |
|---|---|---|---|---|---|---|---|---|---|
| Phi-3.5-mini-instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 12 ms | 975 ms | 1,166 | 0% | 100% | |
| DeepSeek-Coder-V2-Lite-Instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 58 ms | 1244 ms | 789 | 0% | 100% | |
| Llama-3.1-8B-Instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 16 ms | 1949 ms | 1,589 | 0% | 100% | |
| Qwen2.5-Coder-7B-Instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 15 ms | 1141 ms | 944 | 0% | 100% | |
| Qwen2.5-7B-Instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 15 ms | 1699 ms | 1,281 | 0% | 100% | |
| Llama-3.1-70B-Instruct | vllm | 8x NVIDIA H100 80GB HBM3 | 80% | 28 ms | 4338 ms | 1,653 | 0% | 100% | |
| Mistral-7B-Instruct-v0.3 | vllm | 8x NVIDIA H100 80GB HBM3 | 80% | 15 ms | 1243 ms | 824 | 0% | 100% | |
| gemma-2-9b-it | vllm | 8x NVIDIA H100 80GB HBM3 | 100% | 18 ms | 2989 ms | 1,521 | 0% | 100% |
Every signed row carries a hardware fingerprint, software provenance, dataset hash, and seed. Re-verify any envelope:
pip install inferencebench
bench fetch hf://datasets/Yobitel/marathon-keyless-v0.0.2/<file>
bench verify ~/.cache/inferencebench/fetched/*.json \
--require-issuer https://token.actions.githubusercontent.com \
--require-identity-pattern 'github\.com/yobitelcomm/bench/'