How Much VRAM to Run Any LLM (2026 Table)
- Calculate accurately: At 4-bit (Q4) quantization, multiply the model parameter count by 0.5 bytes to find your base weight memory footprint.
- The KV Cache Tax: You must always add roughly 15% to 20% extra VRAM capacity to process dynamic user prompts and expanding context windows.
- Real-world sizing: A robust 70B parameter model utilizing standard compression requires a strict minimum of 48GB VRAM to run natively without offloading.
- MoE constraints: Even though Mixture-of-Experts models activate only small parameter subsets per token, the entire uncompressed file must fit in active memory simultaneously.
Overprovisioning AI hardware drains your CapEx, while underprovisioning crashes your agentic workflows.
To build an efficient local infrastructure, engineering leaders must calculate exact VRAM requirements by model size rather than relying on manufacturer spec sheets.
As established in our main hardware to run local LLMs guide, memory capacity acts as the hard ceiling for inference.
If you cannot fit the weights and the context window into active memory, your system will fail over to CPU offloading, destroying your token generation speed.
The VRAM Calculation Formula (Parameters to Gigabytes)
Model parameter counts are just the starting point. To determine exactly what graphics card you need, you must apply the weight-bit formula.
In uncompressed FP16 (16-bit float), each parameter consumes 2 bytes of memory. Therefore, you multiply the parameter count by two.
However, the 2026 local AI ecosystem runs almost exclusively on quantized models (GGUF, AWQ, EXL2) to maximize hardware efficiency.
FP16 (16-bit): Parameters × 2 bytes
Q8 (8-bit): Parameters × 1 byte
Q4 (4-bit): Parameters × 0.5 bytes
The 20% Overhead Rule: Once you calculate the base weight size, you must add approximately 15% to 20% extra capacity. This overhead is strictly reserved for the KV (Key-Value) cache, which processes your prompt context.
2026 VRAM Requirements Lookup Table
Use this matrix to size your hardware correctly. The figures below represent the total recommended VRAM (Model Weights + 8K Context Window Overhead) required to run inference smoothly without crashing.
| Model Size | Native FP16 (No Quantization) | 8-Bit Quantization (Q8) | 4-Bit Quantization (Q4) | Minimum GPU Target (Q4) |
|---|---|---|---|---|
| 7B | ~16 GB | ~9 GB | ~6 GB | 8GB - 12GB VRAM |
| 13B | ~30 GB | ~16 GB | ~10 GB | 12GB - 16GB VRAM |
| 32B | ~72 GB | ~38 GB | ~22 GB | 24GB VRAM (RTX 3090/4090) |
| 70B | ~160 GB | ~80 GB | ~42 GB | 48GB VRAM (Dual GPU / Mac) |
| 120B | ~270 GB | ~135 GB | ~72 GB | 80GB - 96GB VRAM |
| 671B (MoE) | ~1,400 GB | ~700 GB | ~380 GB | Server Cluster / Cloud API |
If you are specifically tracking Meta's ecosystem for your deployment, you can review our historical baseline for minimum RAM and VRAM requirements for running Llama 4.
Hidden Memory Sinks: KV Cache and Context Windows
Many developers buy a 24GB GPU for a 32B model, load it up, and immediately hit an Out of Memory (OOM) error. They failed to account for the KV cache.
Every token you feed into the prompt, and every token the model generates, must be stored in the KV cache within the VRAM. This cache scales linearly.
While an 8K context window might only consume 1GB to 2GB of VRAM, expanding to a 128K context for massive document analysis can consume an additional 15GB to 20GB of VRAM alone.
Always calculate your expected max context length before purchasing hardware.
Mixture-of-Experts (MoE) Memory Dynamics
Models like DeepSeek R1 or Qwen3-235B utilize a Mixture-of-Experts architecture. MoE models are highly efficient because they only activate a subset of parameters (experts) for any given token.
However, do not confuse active compute with VRAM capacity.
Even if a 235B MoE model only uses 20B parameters of compute per token, you still must load the entire 235B weight file into VRAM simultaneously.
Your tokens-per-second generation will be exceptionally fast, but your base capacity requirement remains massive. To reduce this massive footprint, consult our deep dive on how LLM quantization cuts VRAM. Once your size is mapped and minimized, check out our rankings for the best GPU for local LLM inference.
Conclusion
Accurately predicting your VRAM requirements is the most critical step in building local AI infrastructure. Never purchase hardware based on native FP16 sizes.
By utilizing the 2026 lookup table above, factoring in Q4 quantization, and reserving strict overhead for your KV cache, you can bypass the cloud entirely.
Calculate your exact model footprint today, then source the precise capacity you need.
Frequently Asked Questions (FAQ)
Assuming 4-bit (Q4) quantization and a standard 8K context cache: a 7B model requires roughly 6-8GB of VRAM, 13B requires 10-12GB, 32B demands around 20-24GB, and 70B needs 40-48GB to comfortably fit in active memory.
VRAM relies entirely on parameter count, not the brand. A 70B Llama 4 variant at Q4 requires roughly 40GB to 48GB. A massive 671B DeepSeek R1 model at Q4 will require upwards of 350GB to 400GB of VRAM to run without CPU offloading.
Multiply the parameter count by the precision byte-weight. For FP16, multiply by 2 (e.g., 7B = 14GB). For 8-bit, multiply by 1. For 4-bit, multiply by 0.5. Finally, always add an extra 15% to 20% VRAM capacity to accommodate the KV cache.
Q4 (4-bit) quantization slashes the core model weight memory footprint by roughly 75% compared to native FP16 architectures. This extreme compression makes running robust 70B reasoning models on consumer hardware possible.
The inference engine triggers CPU offloading, spilling the excess model weights into your system's standard DDR RAM. Because system RAM bandwidth is drastically slower than a GPU's VRAM, your tokens-per-second generation speed immediately crashes.
The KV cache scales linearly based on your context length and batch size. A typical 8K context window consumes an additional 1GB to 3GB of VRAM. However, pushing a model to a heavy 128K context can consume over 20GB of VRAM.
While engines like llama.cpp allow you to use system RAM, it is a poor substitute due to severe bandwidth limitations. It prevents hard memory crashes but results in agonizingly slow token generation speeds that are unviable for productivity.
Fine-tuning requires vastly more VRAM than inference because the GPU must map optimizer states, gradients, and forward activations. Expect to need 2 to 4 times the VRAM for LoRA/QLoRA fine-tuning compared to simply running the same quantized model.
Mixture-of-Experts (MoE) models only route data through specific neural pathways per token, requiring less active compute overhead. However, you still must possess enough total VRAM capacity to load the entire multi-billion parameter file into memory simultaneously.
For lightweight 7B to 14B agentic coding assistants, an entry-level 12GB or 16GB GPU is the bare minimum. For serious productivity utilizing 32B or 70B models, aim for a 24GB discrete GPU or a high-capacity unified memory setup.