Phi 3 Mini 3.8B
Microsoft · dense · 3.8B parameters · 131,072 context
Parameters
3.8B
Context Window
128K tokens
Architecture
Dense
Best GPU
A4000
Quality Score
64/100
Intelligence Brief
Phi 3 Mini 3.8B is a 3.8B parameter DENSE model from Microsoft, featuring Multi-Head Attention (MHA) with 32 layers and 3,072 hidden dimensions. With a 131,072 token context window, it supports structured output, code, math. On standardized benchmarks, it achieves MMLU 68.8, HumanEval 47, GSM8K 75. For self-hosted inference, A4000 delivers optimal throughput at $161/month.
Recent changes
Loading…
Related models
5 suggestions- Phi 3 Medium 14BPhi 3 · 14B—
- Phi 3 Small 7BPhi 3 · 7B—
- Phi 2Phi · 2.7B—
- Phi 3.5 MoEPhi · 6.6B$0.300/M out
- Phi 3.5 VisionPhi 3.5 · 4.2B—
Picks: same family first, then same vendor within ±2× params, then top tag-overlap matches. Price shown is the cheapest Output $/M across providers — the row's page shows the canonical anchor.
Architecture Details
Memory Requirements
BF16 Weights
7.6 GB
FP8 Weights
3.8 GB
INT4 Weights
1.9 GB
GPU Compatibility Matrix
Phi 3 Mini 3.8B is compatible with 100% of GPU configurations across 41 GPUs at 3 precision levels.
GPU Recommendations
BF16 · 1 GPU · vllm
100/100
score
Throughput
286.5 tok/s
Latency (ITL)
3.5ms
Est. TTFT
1ms
Cost/Month
$161
Cost/M Tokens
$0.21
BF16 · 1 GPU · vllm
100/100
score
Throughput
458.5 tok/s
Latency (ITL)
2.2ms
Est. TTFT
0ms
Cost/Month
$304
Cost/M Tokens
$0.25
BF16 · 1 GPU · vllm
100/100
score
Throughput
322.3 tok/s
Latency (ITL)
3.1ms
Est. TTFT
1ms
Cost/Month
$237
Cost/M Tokens
$0.28
Deployment Options
API Deployment
No API pricing available
Single GPU
A4000
$161/mo
Min VRAM: 4 GB
Multi-GPU
RTX 3070 x2
409.0 tok/s
TP· $171/mo
API Pricing Comparison
No API pricing data available for this model.
Performance Estimates
Throughput by GPU
VRAM Breakdown (A4000, BF16)
Precision Impact
bf16
7.6 GB
weights/GPU
~286.5 tok/s
fp8
3.8 GB
weights/GPU
int4
1.9 GB
weights/GPU
Quality Benchmarks
Capabilities
Features
Supported Frameworks
Supported Precisions
Where to Deploy Phi 3 Mini 3.8B
Self-Hosted Infrastructure
Similar Models
Phi 3 Small 7B
7B params · dense
Quality: 72
Phi 4 Mini
3.8B params · dense
Quality: 70
from $0.35/M
Minitron 4B
4B params · dense
Quality: 50
from $0.06/M
Nemotron Mini 4B
4B params · dense
Quality: 48
from $0.06/M
Qwen 3 4B
4B params · dense
Quality: 57
from $0.10/M
Frequently Asked Questions
How much VRAM does Phi 3 Mini 3.8B need for inference?
Phi 3 Mini 3.8B requires approximately 7.6 GB of VRAM at BF16 precision, 3.8 GB at FP8, or 1.9 GB at INT4 quantization. Additional VRAM is needed for KV-cache (393216 bytes per token) and activations (~0.50 GB).
What is the best GPU for Phi 3 Mini 3.8B?
The top recommended GPU for Phi 3 Mini 3.8B is the A4000 using BF16 precision. It achieves approximately 286.5 tokens/sec at an estimated cost of $161/month ($0.21/M tokens). Score: 100/100.
How much does Phi 3 Mini 3.8B inference cost?
Phi 3 Mini 3.8B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.