Gemma 4 July 2026 Weight Refresh: Flash Attention 4, Tool-Calling Fixes, and What Your GPU Actually Gets

By RunAIHome Team · 13 min read
gemma-4ollamaflash-attentionlocal-llmgpu

TL;DR: Google refreshed every Gemma 4 checkpoint on July 15–16, 2026 under the same name: Flash Attention 4 prefill gains of 25–70%, tool-calling and truncated-response fixes, and a vision token budget for sharper OCR. The FA4 speedup is datacenter-only (Hopper and newer). Home labs should still re-pull — the agent fixes are the real prize.

What you’ll be able to do after this article:

  • Tell in 60 seconds whether your local Gemma 4 weights are the stale pre-July-16 artifacts, and refresh them in Ollama, LM Studio, or llama.cpp without re-downloading more than you have to
  • Explain why the 25–70% prefill headline does not apply to an RTX 3090, 4090, or even a 5090 — and what the update actually changes on consumer cards
  • Unlock the new 2.51MP OCR resolution where it’s actually exposed (transformers, vLLM, SGLang) and know why Ollama still caps you at the old default

Honest take: If you run Gemma 4 as an agent backend — tool calling from Open WebUI, Cline, or your own scripts — re-pull the weights tonight; the τ2-bench fixes are worth it. If you were hoping your prompts would process 70% faster, they won’t: that number belongs to H100 owners, and your RTX card’s prefill speed is identical to last month’s.

Google’s @googlegemma community post landed July 15, 2026, and by July 16 every Gemma 4 size on Hugging Face — E2B, E4B, 12B, 26B-A4B, and 31B — was serving different weights, kernels, and chat templates than the day before. Same model names, same repo URLs, no version bump. TechTimes called it a stealth update, and that’s the right word: if you pulled Gemma 4 at any point before July 16, 2026, what’s on your disk is materially different from what Google now serves, and none of the fixes activate until you re-pull.

Here’s what shipped, who actually benefits, and the exact update path for a home rig.

What Google actually changed

Four things, per Google’s announcement and the-decoder’s breakdown:

  1. Flash Attention 4 support on NVIDIA Hopper GPUs. Prefill throughput — how fast the model chews through your incoming prompt — improves 25–70%, and time-to-first-token drops up to 31% on H100-class hardware (explainx.ai).
  2. Tool-calling fixes. The refreshed 31B improved on every scenario of the τ2-bench agentic-reasoning suite, with the largest single-task gain of 10.1 percentage points in the telecommunications use case. The inconsistent tool-call JSON formatting that plagued agent frameworks since launch is patched in the chat template.
  3. Truncated-response fix. Google says the refresh cuts down on cases where the model stopped mid-answer or returned incomplete responses — a complaint that had been accumulating in agent workflows since spring.
  4. A vision token budget for OCR. max_soft_tokens — the image token budget for Gemma 4’s vision encoder — can now be raised from its 280 default to 1,120, which lets the model ingest images at up to 2.51 megapixels for noticeably sharper OCR on documents and small text.

The chat template changes matter beyond Google’s own repos. Unsloth and Bartowski re-uploaded fixed GGUF quants, and there’s an active discussion thread on the unsloth/gemma-4-31B-it-GGUF repo titled “New Gemma chat template update by Google” tracking which quants carry the new template. If your GGUF predates mid-July, its embedded template is now wrong.

The FA4 fine print: your RTX card is not invited

Flash Attention 4 is the headline, so let’s be precise about who gets it. FA4 is a kernel rewrite built around hardware that consumer GPUs do not have. Per the FlashAttention compatibility breakdown and Modal’s FA4 analysis:

GPUArchitectureBest FlashAttention tierGets the Gemma 4 prefill boost?
B200 / B300Blackwell datacenter (SM100/SM103)FA4Yes
H100 / H200Hopper (SM90)FA3, FA4 via Google’s Gemma integrationYes — the 25–70% number
RTX 5090Blackwell consumer (SM120)FA2 (no TMEM subsystem)No
RTX 4090 / 4080Ada (SM89)FA2No
RTX 3090 / 3060Ampere (SM86)FA2No

The detail worth underlining: even the RTX 5090 misses out. Consumer Blackwell (SM120) lacks the Tensor Memory subsystem FA4’s tile-based execution model requires, so it runs the same FA2-class kernels as a three-generation-old Ampere card. And your local stack doesn’t use these kernels anyway — llama.cpp and Ollama ship their own flash-attention implementation, which was already on by default for Gemma 4 since Ollama v0.30.0.

So on a used RTX 3090 or an RTX 4090, decode and prefill speeds after the update are the same as before it. The consumer benchmarks that matter haven’t moved: independent testing puts gemma4:26b (the 26B-A4B MoE) at roughly 60–130 tok/s on a 24GB RTX 3090 depending on quant and harness, and the dense 31B at roughly 30–34 tok/s on the same class of card (Markaicode’s Ollama benchmark roundup). Those numbers are why our GPU-by-GPU Gemma 4 guide still recommends the 26B MoE as the sweet spot on 16–24GB cards.

If you genuinely want to feel the FA4 prefill difference — say you’re batch-processing 100k-token documents — the cheapest path is renting Hopper silicon: an H100 PCIe runs $2.89/hr on RunPod’s Secure Cloud, or $1.99/hr on Community Cloud as of July 2026 (pricing comparison). For interactive chat on a home rig, prefill was never your bottleneck; decode bandwidth is, and that story is unchanged.

What home labs DO get: the agent fixes

Strip away the FA4 headline and this update is still worth 10 minutes of your evening, because the tool-calling and template fixes land exactly where local Gemma 4 hurts most.

Since launch, Gemma 4’s tool calls have been a mess of ecosystem workarounds. The symptom set is well documented: tool-call JSON that flips format between turns, agent frameworks failing to parse calls at all — mlx-swift-lm issue #259 (“Gemma 4: tool calls never extracted”) is a representative example, where the parser used outdated tags and inference missed every call. If you’ve wired Gemma 4 into Open WebUI, Cline, or a LangChain agent and watched it randomly drop a tool invocation as plain text, this update is aimed at you. The τ2-bench numbers back it up: improvement on every tested scenario, up to +10.1 points.

The fix reaches your machine through two separate channels, and you likely need both:

Channel 1 — the weights and template on Hugging Face. New checkpoints, new apply_chat_template defaults. Anything that pulls from HF (transformers, vLLM, LM Studio model downloads, GGUF re-quants) gets it on a fresh download only.

Channel 2 — your runtime’s own template and parser. Ollama maintains its own chat templates and tool-call parsing in Go, so it needed a software release: Ollama v0.32.1 (following v0.32.0 on July 11, 2026) specifically improved Gemma 4 tool calling and multi-turn reasoning, including more reliable tool-response continuations (release notes roundup). Running July weights on a June Ollama build leaves half the fix on the table.

This two-channel structure is the same trap we documented in our chat-template troubleshooting guide: the model, the embedded template, and the runtime’s parser all have to agree, and a stale artifact in any one of the three reintroduces the bug.

The update checklist (Ollama)

Total time: about 10 minutes plus download.

1. Update Ollama itself to v0.32.1 or later:

$ ollama --version
ollama version is 0.32.1

Anything older than 0.32.1 predates the Gemma 4 tool-calling improvements. (If you’re still on 0.30.x from spring, you also pick up the v0.32 interactive agent CLI — see our Ollama 0.32 breakdown.)

2. Record your current model digest, then re-pull:

$ ollama list | grep gemma4
gemma4:26b    a91c2f83b6d4    15 GB    5 weeks ago

$ ollama pull gemma4:26b

The digest (that hex column) is your ground truth. If it changes after the pull, you were on stale weights; if the pull says “up to date” and the digest matches, the registry tag hasn’t been refreshed yet — Ollama’s library can lag Hugging Face by days for silent updates like this one, since there’s no new version tag to force a re-index. Check again in a few days rather than assuming you’re current. Stale local caches are the number-one reason people report “the update changed nothing” (Verdent’s Gemma 4 Ollama guide flags the same failure mode).

3. Keep the performance config that already works. The update doesn’t change any of the service-level tuning: OLLAMA_FLASH_ATTENTION=1 (default-on for Gemma 4 since v0.30.0), OLLAMA_KV_CACHE_TYPE=q8_0 to halve KV cache VRAM, and OLLAMA_KEEP_ALIVE to kill cold-start reloads. Set them on the systemd service, not your shell — the full walkthrough is in Ollama Slow? How to Get More Tokens per Second.

4. Re-run your own agent evals. Because the name didn’t change, none of your tooling will warn you that behavior did. If you have a prompt suite for your agent workflows, run it against the refreshed weights before trusting them in anything automated.

For llama.cpp and LM Studio users the equivalent move is re-downloading a post-July-16 GGUF from Unsloth or Bartowski. If you’d rather not re-download 15–18GB for what is partly a template fix, llama.cpp accepts --chat-template-file to override the embedded template with the corrected one — OpenAIToolsHub documents the override path — but you’ll still miss the weight-level τ2-bench gains until you grab fresh quants.

VRAM requirements are unchanged from the June QAT checkpoints: roughly 3GB for E2B, 5GB for E4B, 7GB for the 12B, 15GB for the 26B-A4B, and 18GB for the 31B — the full map is in our Gemma 4 QAT hardware update.

The OCR upgrade is real — but Ollama can’t use it yet

The vision change is the sleeper feature for anyone running Gemma 4 as a local document pipeline. Raising max_soft_tokens from the 280 default to 1,120 lets the vision encoder work at up to 2.51MP, which is the difference between “reads a receipt” and “reads a dense invoice scan.” The supported budget steps are 70, 140, 280, 560, and 1,120.

Here’s the problem for Ollama users: max_soft_tokens is hardcoded to 280 in Ollama’s Gemma 4 image processing code (model/models/gemma4/process_image.go), with no API or Modelfile override. Issue #15626, open since April 16, 2026, asks for exactly this knob and cites a license-plate-recognition test where bumping the budget to 560 in transformers’ AutoProcessor produced correct output that the 280 default missed. Until it’s merged, the sharper-OCR half of this update simply isn’t reachable through Ollama.

Where you CAN use it today: transformers (AutoProcessor exposes max_soft_tokens directly), vLLM, and SGLang. One caveat on bleeding-edge hardware — SGLang has an open bug where Gemma 4 vision prefill collapses at low concurrency with max_soft_tokens=1120 on consumer Blackwell (SM121/GB10), so document-pipeline builders on the newest silicon should pin the 560 budget until that’s resolved. If you’re self-hosting a Gemma 4 document stack, the FOSS deployment side is covered in more depth at aifoss.dev.

The versioning gripe — and why it matters for your benchmarks

The loudest community reaction wasn’t about the features. It was about shipping materially different weights under an unchanged name (the-decoder). The objection is concrete: a benchmark result published against “Gemma 4 31B” on July 10 and one published July 20 describe two different models, and nothing in the model name, repo URL, or tag tells you which artifact produced which number. Critics wanted “Gemma 4.1”; Google treated it like a security patch.

For a home lab this isn’t abstract. Every tok/s figure, every eval score, every “Gemma 4 does X well” claim you read — including ours — now needs a date attached. Our own published Gemma 4 numbers were measured against pre-July-16 checkpoints; throughput won’t have moved on consumer cards for the reasons above, but agentic-quality comparisons (Gemma 4 vs Qwen3.6 as a local coding backend for Cline or Cursor) should be re-run before you trust them. When a vendor won’t version, your notes have to.

FAQ

Do I need to re-download if I only use Gemma 4 for plain chat? It’s still worth it — the truncated-response fix applies to normal generation, not just tool calls — but it’s not urgent. Agent users should update immediately; chat users can wait for their next convenient window.

Will ollama pull gemma4:26b definitely get me the new weights? Not guaranteed on day one. Compare the digest in ollama list before and after pulling. If it’s unchanged, the Ollama library tag hasn’t been re-indexed yet; re-check in a few days. The Ollama-side tool-calling improvements arrive separately via the v0.32.1 app update, so do that half regardless.

Does the update make Gemma 4 faster on my RTX 4090? No. Flash Attention 4 requires Hopper or datacenter-Blackwell silicon; consumer cards including the RTX 5090 run FA2-class kernels, and llama.cpp/Ollama use their own attention implementation regardless. Expect identical tok/s before and after.

Is the 26B still the right size for a 24GB card after the refresh? Yes. Nothing about the memory map changed: the 26B-A4B QAT checkpoint still runs in about 15GB and remains the speed/quality sweet spot at 60–130 tok/s on a 3090-class card. Sizing guidance by VRAM tier is in our models-by-VRAM guide.

How do I know which checkpoint a published benchmark used? For anything published before July 15, 2026: the old weights. After: unknown unless the author says so. That ambiguity is the cost of the silent refresh — assume pre-update numbers for agentic/tool-calling claims and re-verify anything you depend on.

Sources

Last updated July 26, 2026. Prices and specs change; verify current rates before purchasing.

Was this article helpful?