Phi 2
Microsoft · dense · 2.7B parameters · 2,048 context
Parameters
2.7B
Context Window
2K tokens
Architecture
Dense
Best GPU
RTX 3080
Intelligence Brief
Phi 2 is a 2.7B parameter DENSE model from Microsoft, featuring Multi-Head Attention (MHA) with 32 layers and 2,560 hidden dimensions. With a 2,048 token context window, it supports code, math. For self-hosted inference, RTX 3080 delivers optimal throughput at $133/month.
Architecture Details
Memory Requirements
BF16 Weights
5.4 GB
FP8 Weights
2.7 GB
INT4 Weights
1.4 GB
GPU Compatibility Matrix
Phi 2 is compatible with 100% of GPU configurations across 41 GPUs at 3 precision levels.
GPU Recommendations
BF16 · 1 GPU · vllm
100/100
score
Throughput
684.0 tok/s
Latency (ITL)
1.5ms
Est. TTFT
0ms
Cost/Month
$133
Cost/M Tokens
$0.07
BF16 · 1 GPU · vllm
100/100
score
Throughput
244.8 tok/s
Latency (ITL)
4.1ms
Est. TTFT
1ms
Cost/Month
$209
Cost/M Tokens
$0.32
BF16 · 1 GPU · vllm
100/100
score
Throughput
403.2 tok/s
Latency (ITL)
2.5ms
Est. TTFT
0ms
Cost/Month
$85
Cost/M Tokens
$0.08
Deployment Options
API Deployment
No API pricing available
Single GPU
RTX 3080
$133/mo
Min VRAM: 3 GB
Multi-GPU
RTX 3080
684.0 tok/s
Best available config
API Pricing Comparison
No API pricing data available for this model.
Performance Estimates
Throughput by GPU
VRAM Breakdown (RTX 3080, BF16)
Precision Impact
bf16
5.4 GB
weights/GPU
~684.0 tok/s
fp8
2.7 GB
weights/GPU
int4
1.4 GB
weights/GPU
Capabilities
Features
Supported Frameworks
Supported Precisions
Where to Deploy Phi 2
Self-Hosted Infrastructure
Similar Models
Phi 4 Mini
3.8B params · dense
Quality: 70
Phi 1.5
1.3B params · dense
Quality: 38
Phi 1
1.3B params · dense
Quality: 38
RecurrentGemma 2B
2.7B params · dense
Quality: 50
Gemma 2 2B
2.6B params · dense
Quality: 44
Frequently Asked Questions
How much VRAM does Phi 2 need for inference?
Phi 2 requires approximately 5.4 GB of VRAM at BF16 precision, 2.7 GB at FP8, or 1.4 GB at INT4 quantization. Additional VRAM is needed for KV-cache (327680 bytes per token) and activations (~0.30 GB).
What is the best GPU for Phi 2?
The top recommended GPU for Phi 2 is the RTX 3080 using BF16 precision. It achieves approximately 684.0 tokens/sec at an estimated cost of $133/month ($0.07/M tokens). Score: 100/100.
How much does Phi 2 inference cost?
Phi 2 inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.