Open-Source LLM Shootout 2026: Qwen3.6 vs Gemma 4 vs Llama 4 vs GLM-5.1 vs DeepSeek V4 — Which Fits Your GPU?

By RunAIHome Team · 11 min read
local-llmqwen3gemmallama-4deepseekglmmodel-comparisongpuvram

TL;DR: Of the five open-weight families everyone is benchmarking in 2026, only two — Qwen3.6 and Gemma 4 — actually run well on a single consumer GPU. Llama 4 Maverick, GLM-5.1, and DeepSeek V4 Pro are server-class MoEs that need 200 GB+ of VRAM at usable quantization. The headline “five-way shootout” collapses to a two-horse race on the hardware most home labs own, with DeepSeek V4 Flash and Llama 4 Scout sitting in an awkward middle.

Qwen3.6 35B-A3BGemma 4 (12B–31B)Llama 4 Scout 109BDeepSeek V4 Flash 284BGLM-5.1 754B
Best forSingle RTX 3090, speedQuality per GB, 8–24 GBGray-zone, 24 GB at 1.78-bit2× RTX 4090 minimumServer / API only
Min usable VRAM~23 GB (Q3)6.6 GB (12B Q4) → 24 GB (31B Q4)~24 GB (1.78-bit, degraded)~33 GB (heavy quant)~860 GB (FP8)
RTX 3090 speed~120 tok/s30–119 tok/s~20 tok/sNot viableNot viable
LicenseApache 2.0Gemma customLlama 4 CommunityMITMIT

Honest take: If you have one 24 GB GPU, run Qwen3.6 35B-A3B for speed or Gemma 4 31B for reasoning depth — pick by license and task, not by leaderboard rank. The 1T-class models (GLM-5.1, DeepSeek V4 Pro, Llama 4 Maverick) are API models you should use as APIs, not local builds.

The five families, and what they actually are

Every “open-source LLM comparison” article in mid-2026 lists the same five names. The problem is that they treat a 12B dense model and a 1.6T MoE as if they belong in the same buying decision. They don’t. Here is what each family ships, and the size of the gap.

Qwen3.6 (Alibaba). The Qwen3.6-35B-A3B dropped April 16, 2026 — a 35B-total MoE that activates just 3B parameters per token. There are also dense 8B and 14B members carried over from the Qwen3 line. Everything is Apache 2.0, which matters more than people admit. Qwen optimized this family explicitly for consumer hardware.

Gemma 4 (Google). A dense-and-MoE family spanning E2B up to a 31B dense flagship, plus a 26B-A4B MoE. Google’s June 5 QAT (quantization-aware training) checkpoints cut VRAM roughly 72% versus BF16 with near-original quality, which is why a 26B now fits 16 GB. License is Google’s custom Gemma terms — permissive for most use, but not OSI-approved.

Llama 4 (Meta). Scout is 109B total / 17B active; Maverick is 402B total / 17B active. Both are MoE, and both load all parameters into memory regardless of how few activate per token — which is the trap. The Llama 4 Community License carries a 700M-MAU clause, a “Built with Llama” attribution requirement, and a ban on multimodal use by entities domiciled in the EU.

DeepSeek V4. Two checkpoints: V4 Flash (284B total / 13B active) and V4 Pro (1.6T total / ~49B active), released April 2026 under MIT. Flash is the only one with any home-lab pathway.

GLM-5.1 (Z.ai). A 754B open-weight MoE under MIT, released April 7, 2026. It matches frontier closed models on coding benchmarks — and needs roughly 860 GB of VRAM at FP8, i.e. 8× H200. It is on this list because it is excellent, not because you can run it.

Which family fits which GPU

This is the only question that matters for a home lab. Here is the decision by the VRAM you actually have.

8 GB (RTX 3060, RTX 4060, RTX 5060)

Your only real options are small dense models. Gemma 4 12B at Q4 needs about 6.6 GB of weights — it fits, but with little headroom for context, so keep num_ctx modest. A Qwen3 8B at Q4 sits around 5 GB and leaves more room. Llama 4, GLM-5.1, and DeepSeek V4 are all out at this tier — not “slow,” but flatly impossible without spilling to system RAM and crawling.

16 GB (RTX 4060 Ti 16GB, RTX 5060 Ti 16GB, RTX 5070 Ti)

This is where Gemma 4’s MoE shines. The 26B-A4B at Q4_K_M lands right at 16 GB and, because only ~4B parameters activate per token, it generates fast — 64–119 tok/s on a 24 GB RTX 3090, and comfortably usable on a 16 GB card if you trim context. Qwen3 14B dense also fits here. Qwen3.6 30B/35B-A3B at Q4 wants ~17 GB, so it’s right on the edge — doable at low context, more comfortable at 24 GB.

24 GB (RTX 3090, RTX 4090)

The sweet spot, and where the shootout gets interesting:

  • Qwen3.6 35B-A3B: with Unsloth’s Q3 quant it takes ~23 GB and runs at ~120 tok/s on a RTX 3090. This is the fastest capable model you can run on one consumer card.
  • Gemma 4 31B dense at Q4_K_M fills 24 GB and runs ~30–34 tok/s on the 3090 — slower, but a dense 31B scoring 87.1% MMLU is a different quality tier than a 3B-active MoE.
  • Gemma 4 26B-A4B at Q8 also fits 24 GB if you want quality with speed.
  • Llama 4 Scout technically fits via Unsloth’s 1.78-bit dynamic quant at ~20 tok/s — but a 1.78-bit 109B model is a curiosity, not a daily driver. The quality you lose at that bit depth erases the point of running Scout over Qwen3.6.

On a RTX 4090, the same models run faster on the small end — Qwen3 8B Q4 hits ~104 tok/s, 14B ~69 tok/s — but the 24 GB ceiling is identical to the 3090, so it changes speed, not which models fit.

Beyond 24 GB (multi-GPU / server)

DeepSeek V4 Flash needs ~33 GB heavily quantized — realistically 2× RTX 4090 or a single RTX 6000-class card. Llama 4 Maverick at INT4 wants ~200 GB (4× H100). GLM-5.1 and DeepSeek V4 Pro are 8-GPU server territory. If you’re renting rather than buying for these, a cloud GPU like RunPod is far cheaper than assembling an 8× H200 box you’ll use occasionally.

Speed vs. quality: the real trade-off at 24 GB

The MoE-vs-dense split is the whole story on a single 24 GB card. Qwen3.6 35B-A3B activates 3B parameters per token, so it feels like running a 3B model — ~120 tok/s — while having the knowledge of a 35B. Gemma 4 31B is dense: every one of its 31B parameters fires on every token, so you get ~30 tok/s but denser reasoning per token. Neither is “better”; they’re different shapes.

For interactive chat and agentic loops where latency compounds, Qwen3.6’s 120 tok/s wins. For one-shot reasoning, summarization, or code review where you read the output once, Gemma 4 31B’s depth is worth the slower stream. We go deeper on the two-model decision in our Qwen3.6 35B-A3B guide and the Gemma 4 QAT update.

Licenses: the column most comparisons skip

For a home lab tinkering on a side project, license rarely bites. For anyone building a product, it’s decisive.

FamilyLicenseCommercial useCatch
Qwen3.6Apache 2.0Yes, unrestrictedNone
DeepSeek V4MITYes, unrestrictedNone
GLM-5.1MITYes, unrestrictedNone
Gemma 4Gemma customYesUse restrictions, not OSI-approved
Llama 4CommunityYes, with limits700M-MAU clause, “Built with Llama” badge, EU multimodal ban

The irony of the 2026 landscape: the two model families you can actually run locally (Qwen3.6 and Gemma 4) sit at opposite license ends — Qwen3.6 is the cleanest (Apache 2.0), Gemma 4 the most encumbered of the two. If license cleanliness matters to you and you want speed, Qwen3.6 is the unambiguous pick.

The gotcha that wastes an afternoon

The most common failure we see: someone reads “Llama 4 Scout is only 17B active” and runs ollama pull expecting a 17B-class memory footprint. Then they hit CUDA error: out of memory on a 24 GB card. MoE models must hold all 109B parameters in VRAM even though 17B activate per token — the “active” number describes compute, not memory. The fix isn’t a smaller context; it’s accepting that Scout needs either a sub-2-bit dynamic quant (degraded) or more than one GPU. If you hit OOM mid-load on any of these MoEs, check total parameter count, not active count, against your VRAM. (Our full CUDA OOM fixes cover the per-engine flags.)

A second one specific to Gemma 4: do not hand-convert the Hugging Face QAT BF16 weights to llama.cpp Q4_0 yourself — the scale-format mismatch reintroduces the accuracy loss QAT removed. Pull Unsloth’s pre-converted GGUFs or the Ollama -it-qat tags instead.

Ollama availability (June 2026)

If your workflow is ollama pull and go:

  • Qwen3.6 — available, including the 35B-A3B MoE tags.
  • Gemma 4 — available, including QAT tags; the easiest one-command experience.
  • Llama 4 Scout — available but heavy; expect to manage quantization manually.
  • DeepSeek V4 Flash — GGUFs exist (Unsloth), but it’s a vLLM/SGLang job, not a clean Ollama path.
  • GLM-5.1 — Unsloth GGUFs exist; realistically vLLM (v0.10+), SGLang, or KTransformers, not Ollama on consumer hardware.

Honest recommendation by hardware

  • 8 GB card: Gemma 4 12B Q4, or Qwen3 8B if you want more context headroom.
  • 16 GB card: Gemma 4 26B-A4B Q4_K_M — best quality-per-GB at this tier.
  • 24 GB card (3090/4090): Qwen3.6 35B-A3B for speed and a clean Apache 2.0 license; Gemma 4 31B when you need dense reasoning and can live with 30 tok/s. A used RTX 3090 at ~$1,050 (June 2026) remains the value pick here.
  • Anything bigger (GLM-5.1, Llama 4 Maverick, DeepSeek V4 Pro): use the API, or rent cloud GPUs. Don’t buy 200 GB of VRAM for a model you’ll query a few times a week.

For the full per-tier model map beyond these five families, see our best local AI models by VRAM guide, and the focused DeepSeek V4 vs Qwen3 breakdown. Building a coding stack instead? Our sister site aicoderscope.com covers how these models pair with Cursor, Cline, and Continue.dev.

FAQ

Which open model is best for a single RTX 3090 in 2026? Qwen3.6 35B-A3B at Q3 (Unsloth), at ~120 tok/s in ~23 GB. If you value reasoning depth over speed, Gemma 4 31B dense at Q4_K_M (~30–34 tok/s).

Can I run GLM-5.1 or DeepSeek V4 Pro locally? Not on consumer hardware. GLM-5.1 (754B) needs ~860 GB at FP8; DeepSeek V4 Pro (1.6T) is similar 8-GPU server territory. Use them via API.

Does Llama 4 Scout’s “17B active” mean it fits a 17B-class GPU? No. All 109B parameters must sit in VRAM; only 17B compute per token. You need ~24 GB and a sub-2-bit quant to fit one card, with quality loss.

Which has the cleanest license for commercial use? Qwen3.6 (Apache 2.0), DeepSeek V4, and GLM-5.1 (both MIT) have no usage restrictions. Gemma 4 has use restrictions; Llama 4 has the 700M-MAU clause and an EU multimodal ban.

Is a 16 GB card enough for any of these? Yes — Gemma 4 26B-A4B at Q4_K_M lands at 16 GB and runs fast thanks to its 4B active parameters. Qwen3 14B dense also fits.

Sources

Last updated June 15, 2026. Prices and specs change; verify current rates before purchasing.

Was this article helpful?