Best Open Models for 8 GB VRAM in 2026
What actually fits and runs well on an 8 GB GPU: tested quant sizes, real VRAM budgets, and the models we recommend for RTX 4060-class cards.
Best Open Models for 8 GB VRAM in 2026
8 GB cards — an RTX 4060, a used RTX 3060 Ti, a GTX 1070, or an 8 GB Mac — are still the most common hardware people ask us about. The honest answer: 8 GB is enough for genuinely useful models, but you have to pick by file size, not by leaderboard position.
The real 8 GB budget
An 8 GB card never gives you 8 GB for model weights:
| Consumer | Goes to | What is left |
|---|---|---|
| 0.5–1 GB | desktop compositor, browser, video decode | ~7 GB |
| 1.5–2.5 GB | KV cache + activations at 4–8k context | ~5–6 GB for weights |
That is why our GPU detection tool uses a "comfortable" budget of about 72% of VRAM on dedicated GPUs. A 6 GB quantized file is the practical ceiling; a 5 GB file is comfortable.
What fits comfortably (Q4_K_M-class)
| Model | Quant | ≈ File size | Why it wins |
|---|---|---|---|
| Gemma 4 E4B It | Q4_K_M | ~2.3 GB | Best quality-per-GB for assistants |
| Qwen3.5 9B | Q4_K_M | ~5.2 GB | The strongest 8 GB all-rounder |
| LFM2.5 8B A1B | Q4_K_M | ~4.6 GB | MoE speed on weak GPUs |
| Gemma 4 E2B It | Q8_0 | ~2.2 GB | Runs great on CPU too |
Our current pick: Qwen3.5 9B at Q4_K_M. It scores 55.3 on the local.ai Intelligence index — the best number you can reach on 8 GB without going to heavier compression.
What "fits" but we do not recommend
A 12–14B dense model at IQ2/IQ3 fits in 6 GB, but you usually get a model that is worse than a 9B at Q4_K_M. Compression loss beats the extra parameters at this budget. The exception is MoE: Nemotron 3 Nano 30B A3B at IQ2_XXS (~8.1 GB) is borderline — try it, but watch context length.
Quick start with Ollama
ollama pull qwen3.5:9b
ollama run qwen3.5:9b
Check your real VRAM headroom while it runs: nvidia-smi on Windows/Linux.
On Mac, use Activity Monitor → Memory.
Bottom line
Do not buy an 8 GB card for AI in 2026 — a used 12 GB 3060 costs about the same. But if you already own one, Qwen3.5 9B at Q4_K_M is the setup to start with. See the 12 GB guide for what the next step up unlocks.