Updated minutes ago
Zhipu

CogVLM2 19B

THUDM · dense · 19B parameters · 8,192 context

Quality
50.0
Calculate ROI →Compare with othersFine-Tune This Model →

Parameters

19B

Context Window

8K tokens

Architecture

Dense

Best GPU

H100 SXM

Intelligence Brief

CogVLM2 19B is a 19B parameter DENSE model from THUDM, featuring Multi-Head Attention (MHA) with 40 layers and 4,096 hidden dimensions. With a 8,192 token context window, it supports vision, multilingual. For self-hosted inference, H100 SXM delivers optimal throughput at $1794/month.

Architecture Details

TypeDENSE
Total Parameters19B
Active Parameters19B
Layers40
Hidden Dimension4,096
Attention Heads32
KV Heads32
Head Dimension128
Vocab Size128,256

Memory Requirements

BF16 Weights

38.0 GB

FP8 Weights

19.0 GB

INT4 Weights

9.5 GB

KV-Cache per Token655360 bytes
Activation Estimate1.20 GB

Fits on (single GPU) — most practical first

BEST FITRTX 4070 Ti12GBINT4RTX 4070 Super12GBINT4RTX 306012GBINT4A400016GBINT4RTX 408016GBINT4T416GBINT4V100 16GB16GBINT4RTX 4060 Ti 16GB16GBINT4A216GBINT4TPU v5e16GBINT4A10G24GBFP8A3024GBFP8

GPU Compatibility Matrix

CogVLM2 19B is compatible with 76% of GPU configurations across 41 GPUs at 3 precision levels.

BF16 (Full)
FP8 (Half)
INT4 (Quarter)
Blackwell(7 GPUs)
B200 NVL (pair)360GB
89%
95%
97%
B300288GB
87%
93%
97%
B100 SXM192GB
80%
90%
95%
GB200 NVL72 (per GPU)192GB
80%
90%
95%
Hopper(7 GPUs)
H100 NVL 94GB (per GPU pair)188GB
80%
90%
95%
H200 SXM141GB
73%
87%
93%
H2096GB
60%
80%
90%
GH20096GB
60%
80%
90%
Ada Lovelace(11 GPUs)
L40S48GB
21%
60%
80%
L4048GB
21%
60%
80%
RTX 6000 Ada48GB
21%
60%
80%
L2048GB
21%
60%
80%
Ampere(16 GPUs)
A100 80GB SXM80GB
53%
76%
88%
A100 80GB PCIe80GB
53%
76%
88%
A1664GB
41%
70%
85%
RTX A600048GB
21%
60%
80%
Legend:No fitVery tightTightModerateGoodExcellent

GPU Recommendations

H100 SXMoptimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$0.65

Use this config →
H20optimal

FP8 · 1 GPU · tensorrt-llm

100/100

score

Throughput

1.1K tok/s

Latency (ITL)

1.0ms

Est. TTFT

0ms

Cost/Month

$940

Cost/M Tokens

$0.34

Use this config →
H100 PCIeoptimal

FP8 · 1 GPU · tensorrt-llm

95/100

score

Throughput

880.6 tok/s

Latency (ITL)

1.1ms

Est. TTFT

0ms

Cost/Month

$1794

Cost/M Tokens

$0.78

Use this config →

Deployment Options

API

API Deployment

No API pricing available

Self-Hosted

Single GPU

H100 SXM

$1794/mo

Min VRAM: 19 GB

Scale

Multi-GPU

A100 40GB SXM x2

433.6 tok/s

TP· $1613/mo

Open Full Calculator →

API Pricing Comparison

No API pricing data available for this model.

Performance Estimates

Throughput by GPU

H100 SXM
1.1K tok/s
H20
1.1K tok/s
H100 PCIe
880.6 tok/s

VRAM Breakdown (H100 SXM, FP8)

Weights
KV
Act
Weights 19.0 GBKV-Cache 5.4 GBActivations 9.6 GBOverhead 1.0 GB

Precision Impact

bf16

38.0 GB

weights/GPU

fp8

19.0 GB

weights/GPU

~1.1K tok/s

int4

9.5 GB

weights/GPU

Capabilities

Features

✗ Tool Use✓ Vision✗ Code✗ Math✗ Reasoning✓ Multilingual✗ Structured Output

Supported Frameworks

vllmsglangtgi

Supported Precisions

BF16 (default)FP8INT4

Where to Deploy CogVLM2 19B

Similar Models

InternLM 2.5 20B

19.9B params · dense

Quality: 50

from $0.50/M

Larger contextCompare →

GigaChat 20B

20B params · dense

Quality: 50

Similar specsCompare →

Claude 3.5 Haiku

20B params · dense

Quality: 67

from $4.00/M

Larger context, Higher qualityCompare →

GPT-3.5 Turbo

20B params · dense

Quality: 67

from $1.50/M

Larger context, Higher qualityCompare →

InternLM 20B

20B params · dense

Quality: 50

Larger contextCompare →

Frequently Asked Questions

How much VRAM does CogVLM2 19B need for inference?

CogVLM2 19B requires approximately 38.0 GB of VRAM at BF16 precision, 19.0 GB at FP8, or 9.5 GB at INT4 quantization. Additional VRAM is needed for KV-cache (655360 bytes per token) and activations (~1.20 GB).

What is the best GPU for CogVLM2 19B?

The top recommended GPU for CogVLM2 19B is the H100 SXM using FP8 precision. It achieves approximately 1.1K tokens/sec at an estimated cost of $1794/month ($0.65/M tokens). Score: 100/100.

How much does CogVLM2 19B inference cost?

CogVLM2 19B inference costs vary by provider and GPU setup. Use our calculator for detailed cost estimates across all providers.