Updated minutes ago
Zhipu

ChatGLM4 9B

Zhipu AI · dense · 9.4B parameters · 131,072 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

9.4B

Context Window

128K tokens

Architecture

Dense

Best GPU

A100 40GB SXM

Intelligence Brief

ChatGLM4 9B is a 9.4B parameter DENSE model from Zhipu AI, featuring Grouped Query Attention (GQA) with 40 layers and 4,096 hidden dimensions. With a 131,072 token context window, it supports tools, structured output, code, math, multilingual. For self-hosted inference, A100 40GB SXM delivers optimal throughput at $807/month.

Architecture Details

TypeDENSE
Total Parameters9.4B
Active Parameters9.4B
Layers40
Hidden Dimension4,096
Attention Heads32
KV Heads2
Head Dimension128
Vocab Size151,552

Memory Requirements

BF16 Weights

18.8 GB

FP8 Weights

9.4 GB

INT4 Weights

4.7 GB

KV-Cache per Token20480 bytes
Activation Estimate0.80 GB

Fits on (single GPU) — most practical first

BEST FITRTX 40608GBINT4RTX 30708GBINT4RTX 308010GBINT4RTX 4070 Ti12GBFP8RTX 4070 Super12GBFP8RTX 306012GBFP8A400016GBFP8RTX 408016GBFP8T416GBFP8V100 16GB16GBFP8RTX 4060 Ti 16GB16GBFP8A216GBFP8

GPU Compatibility Matrix

ChatGLM4 9B is compatible with 90% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
95%
97%
99%
B300288GB
93%
97%
98%
B100 SXM192GB
90%
95%
98%
GB200 NVL72 (per GPU)192GB
90%
95%
98%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
90%
95%
98%
H200 SXM141GB
87%
93%
97%
H2096GB
80%
90%
95%
GH20096GB
80%
90%
95%
Ada Lovelace(11 GPUs)
L40S48GB
61%
80%
90%
L4048GB
61%
80%
90%
RTX 6000 Ada48GB
61%
80%
90%
L2048GB
61%
80%
90%
Ampere(16 GPUs)
A100 80GB SXM80GB
77%
88%
94%
A100 80GB PCIe80GB
77%
88%
94%
A1664GB
71%
85%
93%
RTX A600048GB
61%
80%
90%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

A100 40GB SXMoptimal

BF16 · 1 GPU · vllm

95/100

score

Throughput

446.6 tok/s

Latency (ITL)

2.2ms

Est. TTFT

0ms

Cost/Month

$807

Cost/M Tokens

$0.69

Use this config →
RTX 5090optimal

BF16 · 1 GPU · vllm

95/100

score

Throughput

514.7 tok/s

Latency (ITL)

1.9ms

Est. TTFT

0ms

Cost/Month

$845

Cost/M Tokens

$0.62

Use this config →
A100 40GB PCIeoptimal

BF16 · 1 GPU · vllm

95/100

score

Throughput

446.6 tok/s

Latency (ITL)

2.2ms

Est. TTFT

0ms

Cost/Month

$655

Cost/M Tokens

$0.56

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

A100 40GB SXM

$807/mo

Min VRAM: 9 GB

Scale

Multi-GPU

A4000 x2

205.1 tok/s

TP· $323/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

A100 40GB SXM
446.6 tok/s
RTX 5090
514.7 tok/s
A100 40GB PCIe
446.6 tok/s

VRAM Breakdown (A100 40GB SXM, BF16)

Weights
Act
Weights 18.8 GBKV-Cache 0.7 GBActivations 6.4 GBOverhead 1.5 GB

Precision Impact

bf16

18.8 GB

weights/GPU

~446.6 tok/s

fp8

9.4 GB

weights/GPU

int4

4.7 GB

weights/GPU

Capabilities

Features

✓ Tool Use✗ Vision✓ Code✓ Math✗ Reasoning✓ Multilingual✓ Structured Output

Supported Frameworks

vllmsglangtgi

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy ChatGLM4 9B

Similar Models

GLM-4 9B

9.4B params · dense

Quality: 50

from $0.15/M

Similar specsCompare →

Gemma 2 9B

9.2B params · dense

Quality: 68

from $0.10/M

Smaller context, Higher qualityCompare →

Eagle 2 9B

9B params · dense

Quality: 65

Smaller context, Higher qualityCompare →

Yi 1.5 9B

8.83B params · dense

Quality: 62

from $0.20/M

Smaller context, Higher qualityCompare →

Yi Coder 9B

8.8B params · dense

Quality: 50

Similar specsCompare →

Frequently Asked Questions

How much VRAM does ChatGLM4 9B need for inference?

ChatGLM4 9B requires approximately 18.8 GB of VRAM at BF16 precision, 9.4 GB at FP8, or 4.7 GB at INT4 quantization. Additional VRAM is needed for KV-cache (20480 bytes per token) and activations (~0.80 GB).

What is the best GPU for ChatGLM4 9B?

The top recommended GPU for ChatGLM4 9B is the A100 40GB SXM using BF16 precision. It achieves approximately 446.6 tokens/sec at an estimated cost of $807/month ($0.69/M tokens). Score: 95/100.

How much does ChatGLM4 9B inference cost?

ChatGLM4 9B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.