Updated minutes ago
ReleasedFebruary 13, 2025source not yet verified
Meta

mmE5-mllama-11b-instruct

intfloat · dense · 10.6B parameters · 131,072 context

Quality
50.0
Deploy this model →Use via API →Calculate ROI →Compare with othersFine-tune this model

Parameters

10.6B

Context Window

128K tokens

Architecture

Dense

Best GPU

H100 SXM

Intelligence Brief

mmE5-mllama-11b-instruct is a 10.6B parameter DENSE model from intfloat, featuring Grouped Query Attention (GQA) with 40 layers and 4,096 hidden dimensions. With a 131,072 token context window, it supports vision. For self-hosted inference, H100 SXM delivers optimal throughput at $1794/month.

Recent changes

Loading…

Related models

4 suggestions

Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.

Architecture Details

TypeDENSE
Total Parameters10.6B
Active Parameters10.6B
Layers40
Hidden Dimension4,096
Attention Heads32
KV Heads8
Head Dimension128
Vocab Size128,256

Memory Requirements

BF16 Weights

21.2 GB

FP8 Weights

10.6 GB

INT4 Weights

5.3 GB

KV-Cache per Token163840 bytes
Activation Estimate0.00 GB

Fits on (single GPU) — most practical first

BEST FITRTX 40608GBINT4RTX 30708GBINT4RTX 308010GBINT4RTX 4070 Ti12GBINT4RTX 4070 Super12GBINT4RTX 306012GBINT4A400016GBFP8RTX 408016GBFP8T416GBFP8V100 16GB16GBFP8RTX 4060 Ti 16GB16GBFP8A216GBFP8

GPU Compatibility Matrix

mmE5-mllama-11b-instruct is compatible with 89% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
94%
97%
99%
B300288GB
93%
96%
98%
B100 SXM192GB
89%
94%
97%
GB200 NVL72 (per GPU)192GB
89%
94%
97%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
89%
94%
97%
H200 SXM141GB
85%
92%
96%
H2096GB
78%
89%
94%
GH20096GB
78%
89%
94%
Ada Lovelace(11 GPUs)
L40S48GB
56%
78%
89%
L4048GB
56%
78%
89%
RTX 6000 Ada48GB
56%
78%
89%
L2048GB
56%
78%
89%
Ampere(16 GPUs)
A100 80GB SXM80GB
74%
87%
93%
A100 80GB PCIe80GB
74%
87%
93%
A1664GB
67%
83%
92%
RTX A600048GB
56%
78%
89%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H100 SXMoptimal

BF16 · 1 GPU · tensorrt-llm

95/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$0.65

Use this config →
H20optimal

BF16 · 1 GPU · tensorrt-llm

95/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$940

Cost/M Tokens

$0.34

Use this config →
A100 40GB SXMoptimal

BF16 · 1 GPU · vllm

95/100

score

Throughput

396.1 tok/s

Latency (ITL)

2.5ms

Est. TTFT

0ms

Cost/Month

$807

Cost/M Tokens

$0.77

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

H100 SXM

$1794/mo

Min VRAM: 11 GB

Scale

Multi-GPU

A4000 x2

184.0 tok/s

TP· $323/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

H100 SXM
1.1K tok/s
H20
1.1K tok/s
A100 40GB SXM
396.1 tok/s

VRAM Breakdown (H100 SXM, BF16)

Weights
Weights 21.2 GBKV-Cache 2.7 GBActivations 0.0 GBOverhead 1.1 GB

Capabilities

Features

✗ Tool Use✓ Vision✗ Code✗ Math✗ Reasoning✗ Multilingual✗ Structured Output

Supported Frameworks

vllm

Supported Precisions

BF16 (default)

Where to Deploy mmE5-mllama-11b-instruct

Similar Models

SOLAR 10.7B

10.7B params · dense

Quality: 50

from $0.30/M

Smaller contextCompare →

Falcon 3 10B

10.3B params · dense

Quality: 50

Smaller contextCompare →

Llama 3.2 11B Vision

11B params · dense

Quality: 9

from $0.18/M

Lower qualityCompare →

Falcon 11B

11B params · dense

Quality: 50

Smaller contextCompare →

Alpamayo 1.5-10B

10B params · dense

Quality: 70

Smaller context, Higher qualityCompare →

Frequently Asked Questions

How much VRAM does mmE5-mllama-11b-instruct need for inference?

mmE5-mllama-11b-instruct requires approximately 21.2 GB of VRAM at BF16 precision, 10.6 GB at FP8, or 5.3 GB at INT4 quantization. Additional VRAM is needed for KV-cache (163840 bytes per token) and activations (~0.00 GB).

What is the best GPU for mmE5-mllama-11b-instruct?

The top recommended GPU for mmE5-mllama-11b-instruct is the H100 SXM using BF16 precision. It achieves approximately 1.1K tokens/sec at an estimated cost of $1794/month ($0.65/M tokens). Score: 95/100.

How much does mmE5-mllama-11b-instruct inference cost?

mmE5-mllama-11b-instruct inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.