TL;DR: 24GB is the tier where the best open models fit with room to think. Qwen3.6-27B — 77.2% SWE-bench Verified, 16.8GB at Q4 — runs with real context headroom instead of on a knife’s edge. Qwen3.6-35B-A3B turns the same card into a 107+ tok/s machine. The catch in August 2026 is the cards, not the models: every 24GB GPU costs more than it did in spring.

	Qwen3.6-27B	Qwen3.6-35B-A3B	Gemma 4 31B QAT
Best for	Coding + hardest tasks — 77.2% SWE-bench Verified	Speed and agents — 107 tok/s on a 3090	Smartest general/multimodal chat
Q4-class footprint	16.8GB	21GB (22.1GB UD-Q4_K_XL)	~18GB (19GB Ollama tag)
Context room on 24GB	32K+ without tricks	8K–16K, KV quantization required	16K–32K with KV quantization
Speed on a used RTX 3090	~40 tok/s (60+ with MTP)	107 tok/s Ollama, 135.7 tuned llama.cpp	Q4-class dense-31B pace, ~25–35 tok/s
The catch	Dense = the slowest of the three	All 35B must fit in VRAM; cache budget is thin	One released quant; don’t convert your own

Honest take: pull Qwen3.6-27B first — it’s the best open coding model that fits on one consumer card, and on 24GB it finally gets the context room a 16GB card denies it. Add the 35B-A3B when you feel the dense model’s ~40 tok/s in agent loops. Gemma 4 31B QAT is the pick if your work is chat, documents, and images rather than code.

The 24GB tier — used RTX 3090, RTX 3090 Ti, used RTX 4090, and AMD’s RX 7900 XTX — has been the enthusiast default for local AI since 2023, and 2026’s model releases were practically aimed at it. The full ladder lives in our VRAM tier guide, the 12GB and 16GB editions cover the rungs below, and the VRAM calculator will check any model + context combination against your exact card before you commit to a 20GB download.

What the step up from 16GB actually buys is different from every previous rung. The 8GB→12GB and 12GB→16GB jumps each bought a smarter model class. The jump to 24GB buys two things at once: the 27B–35B class fits, and — just as important — the 16GB-class models you already know get room to breathe. The same Gemma 4 26B QAT that needs KV-cache surgery on a 5060 Ti runs here with 8GB to spare.

The real VRAM budget on a “24GB” card

Same arithmetic as every tier, bigger numbers. The OS, compositor, and a hardware-accelerated browser take a few hundred MB before you load anything; budget 23 to 23.5GB usable. Into that go the model weights (roughly the GGUF file size), the KV cache that grows linearly with context, and a few hundred MB of compute buffers.

The tier splits into two camps. The 16.8–19GB files (Qwen3.6-27B, Gemma 4 31B QAT, Qwen3-Coder-30B) leave 4–7GB for cache — enough for 32K-class context without heroics. The 21–22GB files (Qwen3.6-35B-A3B at Q4_K_M or Unsloth’s UD-Q4_K_XL) run the tier’s fastest inference but leave under 2GB of slack, which puts you right back in the KV-quantization discipline the 16GB tier taught you.

The picks, by what you actually do

The reason to own 24GB: Qwen3.6-27B with room to think

The dense 27B is the best open-weight coding model that fits on one consumer card: 77.2% on SWE-bench Verified, a score that beats Alibaba’s own 397B MoE and sits in hosted-frontier territory — the full case is in our Qwen3.6-27B guide. At Q4_K_M it’s a 16.8GB file, which on a 16GB card means spillover and compromise. On 24GB it loads fully with ~7GB left over — that’s 32K+ context with an unquantized cache, which is what long agent sessions and whole-repo questions actually consume.

Speed is the honest trade-off. A dense 27B reads all 27B parameters per token, so llama-bench puts a stock RTX 3090 at about 40 tok/s at Q4_K_M, with an RTX 4090 around 70. Multi-token prediction changes the math: InsiderLLM’s config walkthrough gets the same 3090 to 60 tok/s with MTP enabled, and bare-metal setups (no WSL2, no Docker) have reported low 70s. Forty is the floor you plan around; the rest is tuning.

The speed pick: Qwen3.6-35B-A3B

This model is why the phrase “the 24GB line” exists: all 35B parameters must sit in VRAM, so 24GB is its hard floor — and once it’s resident, only ~3B parameters activate per token, so it decodes like a small model. On a used RTX 3090 that’s 107 tok/s through Ollama and 135.7 tok/s with a tuned llama.cpp config; an RTX 4090 reaches ~120 tok/s. VRAM math, context costs, and the tuning flags are in our 35B-A3B guide, with community configs collected in the Compute Market hardware guide and the memorably titled Hugging Face thread “My RTX 3090 ran out of excuses”.

The fine print: the Q4_K_M file is 21GB, so the cache budget is under 2GB unless you quantize it — the trap section below has the exact numbers. And on quality, know what you’re choosing: the dense 27B scores higher on SWE-bench (77.2 vs 73.4). The MoE wins on throughput and long-context economics, not on raw coding IQ.

The general model: Gemma 4 31B QAT

Google’s QAT program gave the 16GB tier its 26B; the 31B flagship is the 24GB tier’s version of the same gift. Per Unsloth’s QAT table it runs in about 18GB — the Ollama QAT tag shows 19GB on disk and lists a 24GB card as the requirement — versus ~20GB for a hand-quantized Q4 with worse quality. It’s the strongest multimodal, most broadly capable chat model that fits this tier, and the same rule from our QAT guide applies: use the released QAT files, don’t convert the base checkpoints yourself.

The agentic coder: Qwen3-Coder-30B-A3B

For Cline, Continue.dev, or any agent that eats context (local-backend setups on aicoderscope.com), the coder-tuned MoE is the practical pick over the dense 27B: ~19GB of Q4_K_M weights, 3.3B active parameters, native 262K context, and ~73 tok/s reported on an RTX 3090 — WillItRunAI clocks a 3090 Ti at 99.5 tok/s and puts the practical total closer to 22–24GB once the cache grows, so treat huge-context runs as a KV-quantization exercise. Architecture background is in our Qwen3-30B-A3B guide. The split: dense 27B for the hardest patches, Coder-30B for long agent sessions at 2× the speed.

And everything from the 16GB tier — GPT-OSS 20B, Gemma 4 26B QAT, Devstral Small 2 — runs here with the context headroom those models never get on their home tier. GPT-OSS 20B at 16K–32K context on a 3090 is a legitimately different tool than the 8K version a 5060 Ti allows.

What you cannot run on 24GB (stop trying)

The ceiling moved up; it didn’t vanish. These are the models 24GB owners keep trying to force, and where each one actually lives:

Model	Q4-class footprint	What it actually needs
Llama 3.3 70B (any dense 70B)	~43–45GB	48GB, or 8–14 tok/s partial offload (the honest math)
Llama 4 Scout	~67GB	Multi-GPU or unified memory
GPT-OSS 120B	72.7GB	Rented cloud
GLM 5.2 (744B MoE)	100GB+ class	Four RTX PRO 6000s
Kimi K3	1.56TB download	Not your house

The dense 70B is the one that hurts. Q2_K at ~26GB still spills over 24GB, and at that quantization a 32B-class model at Q4 beats it on quality anyway. Partial offload works — our 70B-on-24GB guide walks the exact -ngl layer math — but 8–14 tok/s is batch-job speed, not conversation speed. For occasional 70B+ needs, rent: a 48GB A6000-class card on RunPod holds Llama 3.3 70B Q4 fully at 30+ tok/s, and the rent-vs-buy math never favors buying for bursty use.

The trap at this tier: the 21GB file that fits until it doesn’t

Every tier has a signature failure mode. At 24GB it’s this: you pull Qwen3.6-35B-A3B, it flies at 107 tok/s in short chats, then you point an agent at it with 32K context and either hit CUDA out of memory or watch generation collapse to single digits.

The arithmetic is unforgiving. The Q4_K_M weights are 21GB against a ~23.5GB budget. The KV cache on this model costs roughly 1.5GB at 8K context, 3GB at 16K, and 6GB at 32K (Q8_0 cache) — so 8K fits, 16K barely fits, and 32K is 3.5GB past the wall. Nothing warns you at pull time; the failure arrives with the first long prompt. What happens next depends on your OS: llama.cpp errors out cleanly, while a Windows box may silently page VRAM into system RAM — the sysmem-fallback slowdown that ollama ps can’t even see.

The fix order:

1. Quantize the KV cache. OLLAMA_FLASH_ATTENTION=1 + OLLAMA_KV_CACHE_TYPE=q8_0, set on the service, not your shell. Halves the cache cost at near-zero quality loss; details in the tokens-per-second tuning guide.
2. Set context to what the math allows. With a q8_0 cache the 35B-A3B handles 16K comfortably and 32K on a good day. Past that, move the workload to the 16.8GB dense 27B, which does 32K+ without ceremony.
3. Check ollama ps after the first long prompt, not the first short one. The PROCESSOR column must say 100% GPU. A split like 9%/91% CPU/GPU at long context is this exact trap in action.
4. Don’t chase Q5. Q5_K_M on the 35B is a 25.2GB file — it doesn’t fit, full stop. On this tier the quantization quality data says Q4-class with a bigger context beats Q5-class with none.

Which 24GB card in August 2026

The model story at this tier is great; the price story isn’t. The memory shortage that pushed the whole GPU market up this summer hit 24GB cards hardest of all, because AI buyers want exactly these.

Card	Price (Aug 2026)	Bandwidth	Verdict
RTX 3090 used	$1,202–$1,296 asking, ~$1,248 avg	936 GB/s	Still the default. Every number in this article was chosen around it
RX 7900 XTX	~$929 new; used listings ~$815–$950	960 GB/s	The budget play in 2026 — now the cheapest 24GB, with the ROCm caveat
RTX 3090 Ti used	between the 3090 and 4090	1,008 GB/s	4090-class bandwidth at a steep discount when you find one
RTX 4090 used	~$2,268 avg; recent listings to $2,500	1,008 GB/s	The speed ceiling — same models, ~40–75% faster decode, 2026 prices above its old MSRP

The quiet upset of 2026: the used RTX 3090’s price climbed ~72% off its spring lows to ~$1,248, while the RX 7900 XTX lagged the surge — used cards in the $815–$950 range now make it the cheapest ticket into this tier by $300+. The trade is software: ROCm and Vulkan run llama.cpp and Ollama well (our RDNA4 backend comparison covers the landscape), but the CUDA-first corners of the ecosystem — vLLM features, exotic quant formats, day-one support for new architectures — still favor NVIDIA. If your stack is Ollama/llama.cpp and the picks above, the XTX is real value; if you tinker at the edges, the 3090’s premium buys compatibility. The full case for the 3090 is in the value-king analysis, and if you’re power-conscious, power-limiting cuts ~100W for ~3% of the speed.

If you want to run the next tier up

Be honest about what the next rung actually is, because at this tier it isn’t obvious.

The retail answer is the 32GB RTX 5090 — and in August 2026 it’s a terrible deal: the cheapest US listing is $4,381 against a $1,999 MSRP, with medians near $4,700 and climbing. Worse, 8 extra GB doesn’t unlock a new class from here: a dense 70B at Q4 needs ~43GB, so it fits a 5090 no better than it fits your 3090. What 32GB buys from 24GB is comfort — Q5/Q6 quants, two resident models, huge context — not new models.

The real next tier is 48GB, and the affordable route there is a second used RTX 3090 (~$1,248) — two of them hold a 70B at Q4 fully, and our dual-3090 guide covers the PCIe traps before you buy. The GPU buying guide has the full ladder logic. Or skip ownership: a 48GB card on RunPod rents for well under a dollar an hour, which at occasional-use rates outruns a $1,200 purchase for years.

Setup that works, start to finish

Tested against Ollama v0.32.6 (August 4, 2026):

# the coding and hard-tasks default
ollama pull qwen3:27b

# the speed pick (set KV quantization first — see the trap section)
ollama pull qwen3.6:35b-a3b

# the general/multimodal pick
ollama pull gemma4:31b-it-qat

# confirm full GPU residency after a LONG prompt, not a short one
ollama ps
NAME                 ID          SIZE     PROCESSOR    UNTIL
qwen3.6:35b-a3b      e81ac23d    23 GB    100% GPU     4 minutes from now

Run the 27B with --num-ctx 32768 — Ollama’s 2048-token default wastes exactly the headroom you bought the card for. Anything other than 100% GPU in the PROCESSOR column: apply the KV-cache quantization, lower the context, or close whatever else holds VRAM, in that order. For finer offload control through llama.cpp directly, aifoss.dev tracks the open-source tooling.

FAQ

Is a used RTX 3090 still worth it at $1,248? For local AI, yes — grudgingly. It’s 72% off its spring price in the wrong direction, but 936 GB/s + 24GB still has no cheap substitute, and this article’s model list is the payoff. The RX 7900 XTX at ~$900 is the value alternative if your stack is Ollama/llama.cpp.

Qwen3.6-27B vs 35B-A3B — which one first? The 27B. It’s smarter on the benchmarks that matter (77.2 vs 73.4 SWE-bench Verified), and 40 tok/s is past reading speed. Add the MoE when agent loops make you feel the difference between 40 and 107.

Can I run Llama 3.3 70B on 24GB? Partially — 40–45 of 80 layers on GPU, 8–14 tok/s. Fine for overnight batch work, painful for anything interactive. The 70B guide has the exact math and the two better options.

Is the RTX 4090 worth $1,000+ over the 3090? For LLM decode, you’re paying ~80% more for ~8% more bandwidth (1,008 vs 936 GB/s) — decode speed follows bandwidth, so the gap is smaller than the price suggests. The 4090 earns its premium on prefill, image/video generation, and training, not chat throughput.

What about image generation on 24GB? The comfortable tier: SDXL with full headroom, FLUX at FP8 or GGUF Q8 without offload tricks, and video models like Wan become realistic. 24GB is the first tier where LLMs and image generation coexist without model-swapping.

Recommended Gear

• RTX 3090 used — the 24GB default; 936 GB/s and every model on this page
• RX 7900 XTX — the cheapest 24GB in August 2026 for Ollama/llama.cpp stacks
• RTX 4090 used — the speed ceiling for prefill-heavy and image work
• RTX 5090 — the 32GB step up, currently priced for someone else

Sources

• llama-bench: Qwen3.6-27B Q4_K_M ~40 tok/s on RTX 3090 — ahelpme.com
• Qwen 3.6 27B: 60 tok/s with MTP on RTX 3090 — InsiderLLM
• Qwen 3.6 27B VRAM Requirements (16.8GB Q4_K_M) — WillItRunAI
• Qwen 3.6-35B-A3B Local Hardware Guide (107 tok/s Ollama / 135.7 llama.cpp on RTX 3090) — Compute Market
• “My RTX 3090 ran out of excuses: Qwen3.6-35B-A3B” — Hugging Face discussion
• Gemma 4 QAT VRAM table (31B ≈ 18GB) — Unsloth Documentation
• gemma-4-31B-it-qat-GGUF (19GB, 24GB GPU) — Unsloth on Hugging Face
• Qwen3-Coder 30B A3B on RTX 3090 Ti: 99.5 tok/s decode — WillItRunAI
• Qwen3-Coder 30B memory requirements and performance (~73 tok/s on RTX 3090) — Arsturn
• The best local LLMs for RTX 3090 24GB: 2026 ranked guide — PopularAI
• RTX 3090 Used GPU Price & Fair Asking Range ($1,202–$1,296, avg $1,248) — ResalePrices
• RTX 4090 Price Tracker US, Aug 2026 (~$2,268 used) — Best Value GPU
• RX 7900 XTX Price Tracker US, Aug 2026 — Best Value GPU
• Best 24GB VRAM GPU for Local LLMs: bandwidth comparison — LLMHardware.io
• RTX 5090 at $4,381+ as market prices pass $4,300 — BigGo Finance
• RTX 50-series prices spike further above MSRP in August — ThinkComputers
• Ollama Release Notes, v0.32.6 — releases.sh

Last updated August 12, 2026. Prices and specs change; verify current rates before purchasing. Tokens/sec figures vary with runtime, quantization, context length, and tuning; treat the low end of each range as the planning number. The Gemma 4 31B QAT speed is a bandwidth-class estimate, not a measured benchmark.