How much VRAM do you need to run an AI model locally?

The single most common question from people trying local AI for the first time is some version of "will this run on my computer?" The answer almost always comes down to one number: how much memory your graphics card has. Here is how to work it out without downloading anything.

Three things take up memory

First, the weights. A model with 8 billion parameters stored at 4 bits (Q4) needs 8 x 0.5 = 4 GB. At 8 bits it needs 8 GB, and at full FP16 precision 16 GB. Second, runtime overhead: buffers and scratch space add roughly 10%. Third, the KV cache, which grows with how long your conversation is. For an 8B model at 8K tokens of context that is about 1 GB; for a 70B model it is closer to 2.6 GB.

Add them up and an 8B model at Q4 with 8K context needs around 5.5 GB. That fits on an 8 GB card with room to spare, and on a 6 GB laptop card if you keep context short.

Not all of your VRAM is usable

Windows, your browser and the display itself take a slice. Plan on about 90% of the printed VRAM being available. A 12 GB RTX 3060 gives you around 10.8 GB to work with. On Apple Silicon the memory is shared with everything else the Mac is doing, and macOS caps what the GPU can claim, so budget about 72% of the unified memory.

Quick reference

Your VRAM	Comfortable at Q4	Possible with care
4 GB	1B to 3B models	7B at Q4 with 2K context, slow
6 to 8 GB	7B to 9B	12B to 14B at Q4, short context
12 GB	12B to 14B	Qwen3 30B-A3B MoE, 24B at Q3
16 GB	20B to 27B, gpt-oss 20B	32B at Q3
24 GB	27B to 32B	70B at Q2, barely
32 GB	32B at Q6, 30B MoE at Q8	70B at Q3
48 GB and up	70B at Q4	120B MoE models

What if it doesn't fit?

Tools like Ollama and llama.cpp can place some layers in system RAM and run them on the CPU. It works, but generation speed drops from a brisk 30 tokens per second to a painful 2 to 5. For chat that feels broken. For a one-off batch job overnight, it is fine. The checker marks these cases separately so you know what you are getting into.