RunLLMLocal
NewQwen3, gpt-oss and Gemma 3 added · Oct 2026

Can my PC run this LLM?

Pick your graphics card or Mac. See which open AI models fit in memory, which quantization to download, and how fast they will run.

32open models
41GPUs
15Apple chips

Models 0 of 0 run

● Runs in your browserMemory = weights × 1.1 + KV cache. Usable = 90% VRAM, 72% on Mac.

How it works

Know what runs before you download 40 GB.

Local AI models live in your graphics card's memory. If the model file plus its working memory is bigger than your VRAM, it either crawls on the CPU or refuses to load. RunLLMLocal does that arithmetic for every popular open model, in your browser, before you install anything.

Pick your card or Mac chip, set how much context you want, and get a ranked list: what runs well, what runs with CPU offload, and what won't fit. Each result shows the quantization that fits, the memory it needs and a rough speed in tokens per second.

Real memory mathWeights at the chosen quantization, 10% runtime overhead, plus a KV cache sized to your context length. The same formula llama.cpp users work out by hand.
Speed estimateTokens per second derived from your card's memory bandwidth and the model's active parameters. Rough, but in the right ballpark.
Mac awareApple Silicon shares memory between CPU and GPU. We use the realistic 72% that macOS lets a model take.
Offload, not just yes/noA model that overflows VRAM can still run by splitting layers to system RAM. We show when that is worth it and when it isn't.

Questions

What is quantization and why does it matter?

Model weights are stored as numbers. FP16 uses 2 bytes per weight, Q8 uses 1, Q4 uses half a byte. Q4 cuts memory by 75% with a small quality loss, which is why most people run Q4 or Q5 at home.

How accurate are the speed numbers?

They are estimates from memory bandwidth, the main limit for single-user inference. Real numbers vary with your software (Ollama, LM Studio, llama.cpp), drivers and context size. Expect plus or minus 30%.

Why does context length change memory?

Every token in the conversation keeps a key/value entry in memory. At 32K tokens that cache can be several gigabytes for larger models. Shorter context frees room for a bigger model.

Can I run a model bigger than my VRAM?

Yes, by offloading some layers to system RAM. It works, but speed drops sharply, often to 1 to 5 tokens per second. We mark these as "Runs with offload".

Where do I actually download these models?

Ollama (ollama.com) and LM Studio (lmstudio.ai) both let you pick a model and quantization and handle the download. Hugging Face hosts the raw GGUF files.

Copied