Methodology

This page states exactly what the VRAM calculator computes, what data is exact versus estimated, and where the estimate can be wrong. If a number on this site doesn't match your real-world results, this page is where to check why before assuming either number is broken.

The three VRAM components

Every estimate on this site is the sum of three terms:

Model weights — either exact (a real synced .gguf file size from Hugging Face) or estimated as parameter_count × (bits_per_weight / 8) in generic mode.
KV cache — always estimated, using the formula below. This is the term most people underweight when sizing hardware.
Runtime overhead — a flat 10% of (weights + KV cache), a rule-of-thumb margin for CUDA context and activation buffers, not a measured figure.

The exact KV cache formula

kv_bytes = 2 × layers × kv_heads × head_dim × context_tokens × 2

The first 2 accounts for storing both keys and values. The final 2 is bytes per value at fp16 — the calculator does not model quantized KV cache, which some inference engines support as a separate setting. layers, kv_heads, and head_dim come from the matched reference architecture below, not from the specific model you selected.

The reference architectures

Four fixed architecture buckets back every KV cache estimate on this site. 7B, 13B, and 70B are real, published model specs. 120B has no real published reference model at that exact size and is extrapolated from the 70B architecture — not a measured spec.

32 layers · 8 KV heads · 128 head dim

13B

40 layers · 40 KV heads · 128 head dim

70B

80 layers · 8 KV heads · 128 head dim

120B

88 layers · 8 KV heads · 128 head dim

GQA vs. MHA, and what this site actually does with it

Grouped-Query Attention (GQA) shares key/value projections across groups of query heads, which shrinks KV cache substantially versus classic Multi-Head Attention (MHA) at the same parameter count. The 7B and 70B reference buckets above use GQA-style numbers (matching Llama 3 8B and Llama 3 70B); the 13B bucket uses MHA-style numbers (matching Llama 2 13B), which is deliberately heavier on KV cache to represent an older architecture still in common use.

This site does not detect whether your specific model is actually GQA or MHA. For a model synced from Hugging Face, the calculator parses an approximate parameter count out of the model's name (for example, "8B" from "Meta-Llama-3-8B-Instruct") and matches it to the nearest bucket by size alone — not by reading the model's real config for num_attention_heads versus num_key_value_heads. If a specific model at, say, 8B parameters happens to use MHA instead of GQA, this calculator will still estimate its KV cache using the 7B bucket's GQA numbers, and the real figure would be meaningfully higher. This is a known, current limitation — the heuristic is "close for most current models at common sizes," not "verified per model."

What's exact vs. estimated, summarized

Exact

Synced model weight file size (model-specific mode only)

Estimated

Generic mode weight size
KV cache, in both modes, always
Runtime overhead (flat 10% rule of thumb)
Which reference architecture applies to a given synced model