Local Coding Models by Memory
From 4 GB to 512 GB, the right local coding model is the one that leaves enough memory for context, KV cache, and the runtime—not the largest file that barely loads.
Most local LLM guides answer the easiest question: what is the largest model file that fits in memory?
That is not the question I care about.
I want to know which model can read the code, hold a useful conversation, run tools, survive a few failed tests, and still have enough memory left to finish the task. A model that consumes the entire card before the first prompt is not a working setup. It is a successful loading screen.
I built this list from Cloud Codes' video, Best Local Coding AI for Your GPU (4GB to 512GB), then checked the linked model cards and current quantization sizes. The exact winner will keep changing, but the memory math will not.
The short version
These are the configurations I would start with today. Weight sizes are approximate and do not include the KV cache or runtime overhead.
| Memory | Daily-driver pick | Quant / weight target | What it is good for |
|---|---|---|---|
| 4 GB | Spark-X2.5-4B | 4-bit, about 2.6 GB | Explaining code, one focused bug, small edits |
| 6 GB | NeoHorse-1-4B | Q5_K_M, 3.07 GB | Focused coding with better tool use |
| 8 GB | Ornith-1.5-9B | 4-bit, about 5.75 GB | One file plus a useful conversation |
| 12 GB | Ornith-1.5-9B | 8-bit, about 9.75 GB | The same model with cleaner output; still keep context lean |
| 16 GB | Qwen3.8-27B GSQ-RCO | IQ3_S, 11.8 GB | Real coding, limited multi-file work, 16K–32K context |
| 24 GB | Qwen3.8-27B | Good 4-bit / low-5-bit build, about 16.5 GB | Multi-file changes, tests, longer conversations |
| 32 GB | Qwen3.8-27B | 6-bit, about 22 GB | High-quality 27B with comfortable working room |
| 48 GB | Qwen3-Coder-Next | Compact IQ4/dynamic quant, about 38 GB | Long agent loops, editing, testing, recovery |
| 64 GB | Qwen3-Coder-Next | Official Q4_K_M, 48.4 GB | The practical autonomous-agent tier |
| 80 GB | Qwen3-Coder-Next | Q6_K, 65.5 GB | Better fidelity with room for a large context |
| 96 GB | Qwen3-Coder-Next | Q8_0, 84.8 GB | Near-lossless coding agent on one large card |
| 128 GB | Qwen3.8-Flash-Next | Q4-class, about 111 GB | Large sparse model with enough room to work |
| 192 GB | GLM-5.3-Flash | Lower 4-bit build, roughly 162 GB | Frontier-class coding without filling the whole machine |
| 256 GB | MiniMax-M3 | IQ4_XS, about 208 GB | 428B total / 23B active; strong all-day sparse-agent setup |
| 512 GB | GLM-5.3 | 3-bit, about 343 GB | The sensible daily driver; Q4 at 467 GB is the maximum-quality option |
There are two places where I would resist the obvious upgrade.
At 12 GB, I would rather run a good 9B model at 8-bit than squeeze in a larger model with almost no context. At 512 GB, I would rather run GLM-5.3 at a good 3-bit quant with more than 150 GB free than load the 467 GB Q4 build and pretend the remaining memory is generous.
The memory nobody puts in the model name
Three things compete for the same memory:
- Weights — the GGUF or other model files.
- KV cache — the model's working notes for every token already in the conversation.
- Runtime scratch space — temporary buffers, graphs, tool state, multimodal projectors, and sometimes a speculative-decoding model.
The rough budget is:
usable model memory = total memory - system use - weights - KV cache - runtime buffers
For a standard attention layer, KV-cache memory grows roughly like this:
tokens × layers × KV heads × head size × 2 (K + V) × bytes per value × parallel slots
The exact number depends heavily on the architecture. Grouped-query attention reduces the number of KV heads. Sliding-window, recurrent, and hybrid attention can reduce how much history must stay resident. Quantized K and V caches reduce it again. Multiple concurrent slots multiply it.
DeepSeek-V4.1-Flash shows how much the architecture can change the equation. DeepSeek reports a global KV-cache footprint of 890 bytes per token—about one quarter of its previous Flash model—using shared sparse-attention state and FP4 KV caching. A million-token context is still expensive, but it no longer has to scale like a conventional dense Transformer.
The important part is simpler: context is not free, and its cost grows with the context window you allocate.
This matters in llama.cpp because the configured context size reserves cache capacity. If I start a server with -c 262144, I am budgeting for a 262K-token cache even when the first prompt is tiny. On a 16 GB card, dropping to -c 32768 can be the difference between a useful 27B model and an out-of-memory error.
My starting rule is to leave at least:
- 1.5–3 GB free on 4–12 GB devices
- 4–8 GB free on 16–32 GB devices
- 12–24 GB free on 48–128 GB devices
- 10–20% free on larger unified-memory or multi-GPU systems
Those are planning numbers, not laws. The real test is peak memory after loading the model, ingesting a representative repository context, and completing several agent steps.