Most LLM memory calculators make two quiet mistakes. They ask you to type a parameter count you are guessing at, and they size the KV cache from num_attention_heads. The second one inflates the cache badly on modern models. Under grouped query attention the tensors that persist across decoding are the keys and values, and their count comes from num_key_value_heads, a different config field that is usually several times smaller. Pope et al. show that sharing key and value heads across query heads lowers memory requirements and lets inference scale to much longer contexts (arXiv 2211.05102).
Take Qwen2.5 7B. Its published config.json sets "num_attention_heads": 28 and "num_key_value_heads": 4. Open the file and check. A calculator that sizes the cache from the attention head count over-reports it by 7x for this model. ModelScope, an independent host run by a different operator, serves the same two values in its copy of the file, so this is not a typo on one hub.
The parameter counts here are fetched, not typed. Every preset carries the exact count from the safetensors.total field of that model's Hugging Face model-info record. Llama 3.1 8B, for example, reports 8030261248 parameters, which is 8.03 billion, a little over the name. Pick a preset and both numbers land in the calculator for you.
Every result recomputes as you type, in your browser. Weights are the parameter count times the bytes per parameter you pick. The KV cache is per-token bytes times context length times batch, held at fp16, two bytes per value, doubled once because K and V are separate tensors (Pope et al., arXiv 2211.05102; Kwon et al., arXiv 2309.06180). Adam's optimizer states cost eight bytes per parameter, two fp32 moment estimates of four bytes each (Kingma and Ba, arXiv 1412.6980; Micikevicius et al., arXiv 1710.03740). The activation line is an estimate and says so, with its coefficient printed right next to it.
Copy these from your model's config.json on the Hub.
| Line | Memory | How the number is computed |
|---|
For a model outside the list, pick Custom and copy num_hidden_layers, num_key_value_heads and head_dim from its config.json. When head_dim is not published, divide hidden_size by num_attention_heads. Qwen2.5 7B, for instance, publishes "hidden_size": 3584 alongside its 28 attention heads, and the head dimension follows from that division.
One row per preset. The parameter column is the exact safetensors.total from the linked record, in billions. Weights columns use two bytes per parameter at fp16 and half a byte at int4. Adam states use eight bytes per parameter (Micikevicius et al., arXiv 1710.03740). The KV column assumes an fp16 cache, a single sequence, and a 128k context. And the over-report column is the reason this page exists. It shows how far a num_attention_heads calculator overstates that model's cache.
| Model | Params (B) | Q heads per KV head | Naive KV over-report | Weights fp16 (GiB) | Weights int4 (GiB) | Adam states (GiB) | KV cache at 128k (GiB) | Count source |
|---|---|---|---|---|---|---|---|---|
| Llama 3.1 8B | 8.03 | 4 | 300% | 14.96 | 3.74 | 59.83 | 16 | record |
| Llama 3.2 1B | 1.24 | 4 | 300% | 2.3 | 0.58 | 9.21 | 4 | record |
| Qwen2.5 0.5B | 0.49 | 7 | 600% | 0.92 | 0.23 | 3.68 | 1.5 | record |
| Qwen2.5 1.5B | 1.54 | 6 | 500% | 2.88 | 0.72 | 11.5 | 3.5 | record |
| Qwen2.5 3B | 3.09 | 8 | 700% | 5.75 | 1.44 | 22.99 | 4.5 | record |
| Qwen2.5 7B | 7.62 | 7 | 600% | 14.19 | 3.55 | 56.74 | 7 | record |
| Qwen2.5 14B | 14.77 | 5 | 400% | 27.51 | 6.88 | 110.05 | 24 | record |
| Qwen2.5 32B | 32.76 | 5 | 400% | 61.03 | 15.26 | 244.11 | 32 | record |
| Qwen2.5 72B | 72.71 | 8 | 700% | 135.43 | 33.86 | 541.7 | 40 | record |
| Qwen3 8B | 8.19 | 4 | 300% | 15.26 | 3.81 | 61.03 | 18 | record |
| Qwen3 14B | 14.77 | 5 | 400% | 27.51 | 6.88 | 110.03 | 20 | record |
| Qwen3 32B | 32.76 | 8 | 700% | 61.02 | 15.26 | 244.1 | 32 | record |
| Mistral 7B v0.1 | 7.24 | 4 | 300% | 13.49 | 3.37 | 53.96 | 16 | record |
| Mistral NeMo 12B | 12.25 | 4 | 300% | 22.81 | 5.7 | 91.25 | 20 | record |
| Mixtral 8x7B | 46.7 | 4 | 300% | 86.99 | 21.75 | 347.96 | 16 | record |
| Phi-3 mini 4k | 3.82 | 1 | 0% | 7.12 | 1.78 | 28.47 | 48 | record |
| Phi-4 | 14.66 | 4 | 300% | 27.31 | 6.83 | 109.22 | 25 | record |
| OLMo 2 7B | 7.3 | 1 | 0% | 13.59 | 3.4 | 54.38 | 64 | record |
| SmolLM2 1.7B | 1.71 | 1 | 0% | 3.19 | 0.8 | 12.75 | 24 | record |
The ratio is not one number across model families. Mistral 7B v0.1 publishes "num_key_value_heads": 8 against "num_attention_heads": 32 in its config.json. phi-4 publishes "num_key_value_heads": 10 against "num_attention_heads": 40 in its file. And OLMo 2 7B keeps full multi head attention, with "num_key_value_heads": 32 equal to its "num_attention_heads": 32 in its config, so the naive method happens to be right there. You cannot assume the correction away. Check the config, or use a calculator that already did.
Llama 3.1 8B reports 8030261248 parameters in safetensors.total (the model-info record). Serve it at fp16 and the weights take 14.96 GiB, the count times two bytes. The cache then adds 131072 bytes for every token in flight, 128 KiB each, so one full 128k-token sequence costs 16 GiB on top of the weights. Fine-tune the same model with Adam and the bill changes shape. Gradients at fp16 match the fp16 weights, another 14.96 GiB, and the optimizer's two fp32 moment tensors add 59.83 GiB at eight bytes per parameter (Micikevicius et al., arXiv 1710.03740). Mixed precision training also maintains a single-precision master copy of the weights that accumulates the gradients after each optimizer step, so budget four more bytes per parameter if your framework keeps one. Even with that, the same paper reports mixed precision training can reduce the memory consumption of deep learning models by nearly 2x against full fp32.
Kwon et al. put it plainly in the vLLM paper. The KV cache memory for each request is huge, and it grows and shrinks dynamically (arXiv 2309.06180). So at long contexts the context length input moves your total more than any other field on this page. Sizing the cache precisely, per model and per quantization, is its own topic, and this site already covers it in depth. The KV cache size calculator goes further than the summary here.
The other calculator probably multiplied by num_attention_heads. Grouped query attention stores keys and values for num_key_value_heads only, and the query heads share them (Pope et al., arXiv 2211.05102). The over-report column in the table shows the gap per model.
Adam keeps adaptive estimates of lower-order moments of the gradients, two state values per parameter (Kingma and Ba, arXiv 1412.6980). Held in fp32 at four bytes each, that comes to eight bytes per parameter, four times what the fp16 weights themselves cost. The optimizer, not the model, dominates a training budget.
No, only the weights line. Quantizing weights to half a byte per parameter leaves this page's fp16 KV cache and the activation estimate untouched. At long contexts the cache can outweigh the quantized weights, and the table's 128k column makes that easy to spot.
Runtimes differ in what they keep live. Fused kernels, activation recomputation, memory pooling and batching strategy all change the real footprint, so the calculator states its coefficient (two live values of d_model per layer per token, at two bytes each) instead of pretending to measure. Trust a profiler over any formula, this one included.
Micikevicius et al. report that mixed precision training can reduce the memory consumption of deep learning models by nearly 2x, with weights, activations and gradients stored in IEEE half-precision format, sixteen bits per value (arXiv 1710.03740). The savings land in the weights and gradients lines here. The fp32 master copy and the Adam states stay full width.
Nothing on this page is typed from memory. Parameter counts are read from each model's Hugging Face model-info record, the safetensors.total field, then cross checked against the model's safetensors byte footprint divided by its stored dtype width, and against the same footprint on ModelScope, an independent host. Per-token KV figures come from each model's own config.json, recomputed in code as two bytes at fp16, times two for K and V, times num_key_value_heads, times head_dim, times num_hidden_layers, following Pope et al. (arXiv 2211.05102) and Kwon et al. (arXiv 2309.06180). A value the two hosts disagree on is held back, and nothing gets published for it. Models whose attention layout the formula does not describe are excluded rather than approximated, so no preset carries a silently wrong number. Weights, gradient and optimizer results are pure multiples of the fetched count by the bytes-per-parameter constant shown beside each one, recomputed in your browser so you can reproduce every figure.
Every ml0x tool runs client side, and I check that the way you would, by watching the network panel while the page works. I ran that inspection across the live ml0x calculator pages on 10 July 2026 and the computation stayed on the machine. This page is built the same way.