LLM GPU memory calculator

A quick, free estimate of how much GPU memory an open-weight language model needs to run, and how many GPUs of a given size that means. Pick a model, set a few options, and the answer updates as you go.

1. Model Pick an open-weight model, or choose "Custom" and enter its size.

Model size (billions of parameters) For example 8 for an 8B model. KV cache is estimated from size when the architecture is unknown.

2. Weight precision Lower precision (quantization) uses less memory with a small quality trade-off. 16-bit is the safe default. 3. Context length (tokens per conversation) How much text the model keeps in view: prompt plus answer. 8,192 is a common starting point. 4. Simultaneous users Conversations running at the same moment, each with the full context above. 5. KV cache precision The model's working memory per conversation. 8-bit halves it and is widely supported. 6. Memory per GPU The memory on each GPU you're considering, in GB. The result tells you how many you'd need.

Estimated GPU memory needed

–

Weights KV cache Overhead

How to read the result

Weights are the model itself. They're loaded once, whatever the number of users.
KV cache is the working memory for each conversation. It grows with context length and with the number of simultaneous users, and it's usually what pushes you onto more GPUs.
Overhead covers the runtime and temporary buffers: we allow 10% of the weights plus 1 GB per GPU, and keep 10% of each GPU free.
When a model needs more than one GPU, serving software normally splits it across 2, 4 or 8 GPUs, so we also show the next of those counts.

How we calculate (and the limits)

Weights = parameters × bytes per parameter (2 for 16-bit, 1 for 8-bit, about 0.55 for 4-bit including quantization scales).
KV cache = layers × cached values per layer × bytes × context length × simultaneous users, using each model's published architecture.
Mixture-of-experts models still need memory for all their parameters, even though only some are active per token.
These are planning estimates, not guarantees. Real usage varies with the serving software, settings and the exact quantization format. Test before you buy.
This calculator doesn't recommend any vendor or product. It answers one question: how much memory, and how many GPUs of the size you chose.

Model list last updated: –. We refresh it weekly.

Sizing is the easy part. Picking the right workflow is harder.

Our free 3-minute assessment tells you whether AI is worth building into your firm yet, and where to start.

Take the free assessment