TL;DR: A local model that loops, rambles, or prints GGGGGG almost always has one of three problems — a wrong chat template, bad sampling settings, or a broken quant/backend. Fix the template first (it causes the most “gibberish” reports), then tune sampling, and only blame the file last. Cranking repeat penalty higher usually makes looping worse, not better.

What you’ll be able to do after this:

• Tell the difference between a template bug, a sampling problem, and a corrupt quant in under two minutes.
• Stop repetition loops the right way — with DRY and sampler order, not a heavy-handed repeat penalty.
• Know when your repeat_penalty setting is being silently ignored (yes, that’s a real Ollama bug in 2026).

Honest take: 90% of “my model outputs garbage” posts are a chat-template mismatch or a repeat penalty set too high. Check those two before you re-download a single byte.

First, figure out which failure you actually have

These three look similar in the terminal but have completely different causes. Read your output and match it:

Symptom	Most likely cause	Where to look
Coherent text that loops (“the cat sat sat sat” or repeats a paragraph)	Sampling: penalty too high/too low, or context overflow	Sampling params
Output that starts fine then leaks <start_of_turn>, `<	im_end	>`, or ignores your prompt
Pure garbage from token one — GGGGGGGG, !!!!!, or non-language symbols	Broken quant, bad backend, or unsupported hardware path	The GGUF file + backend

If you can’t tell, run the model with a dead-simple prompt like Say hello. at temperature 0. Coherent-but-looping is a sampling problem. Garbage from the first token is a file/backend problem. Template literals in the output are a template problem.

Fix 1: The wrong chat template (the #1 cause of “gibberish”)

Every GGUF embeds a Jinja2 chat template — the exact wrapper that turns your message into the <start_of_turn>user … <start_of_turn>model tokens the model was instruction-tuned on. When a runner applies the wrong template, the model never sees a properly formatted prompt, so it improvises: endless generation, repeated output, or literal control tokens bleeding into the text. This is the single most common reason a model “works fine in one tool and spews nonsense in another.” (Unsloth GGUF docs, Ollama issue #2334)

Tell-tale signs: you see <start_of_turn>, <end_of_turn>, <|im_start|>, or <|eot_id|> literals in the reply, or the model answers a question you didn’t ask, or it never stops. Gemma 4 GGUFs shipped with a broken template early in their release and produced exactly this until re-uploaded (Gemma 4 GGUF chat template fix — OpenAIToolsHub).

Ollama — the fastest fix is to re-pull from the official library, which carries a corrected template:

$ ollama rm gemma4:12b
$ ollama pull gemma4:12b   # official tags get template patches; random HF imports don't

If you imported a raw GGUF with a Modelfile, the template is your responsibility. A minimal Gemma-style Modelfile:

FROM ./gemma-4-12b-it-Q4_K_M.gguf
TEMPLATE """<start_of_turn>user
{{ .Prompt }}<end_of_turn>
<start_of_turn>model
"""
PARAMETER stop "<end_of_turn>"

Note the stop parameter — small and mid-size models will hallucinate extra turns forever without an explicit stop token.

llama.cpp — override the embedded template instead of re-downloading a 20 GB file:

llama-server -m model.gguf --chat-template-file ./corrected_template.jinja

LM Studio — it auto-applies the embedded template; to override, edit the model’s preset JSON (the “Prompt Template” field) and paste the correct format. If you don’t know the right one, the model card on Hugging Face lists it.

Before you touch anything else, confirm this isn’t your problem. It usually is.

Fix 2: Runaway repetition (and why more penalty makes it worse)

If the text is coherent but loops, it’s sampling. Here’s the counterintuitive part that trips up almost everyone: turning the repeat penalty up often triggers more looping, not less. Unsloth’s Daniel Han documented this directly for QwQ-32B — heavy repetition penalties push the model into degenerate loops, and the fix is to lean on the DRY sampler and fix sampler order instead (Daniel Han / Unsloth QwQ-32B guide).

A few facts about llama.cpp defaults that matter here: repeat_penalty defaults to 1.0 (disabled), repeat_last_n defaults to 64, and the DRY multiplier defaults to 0.0 (off). llama-server does not enable repeat penalty by default — so if you’re getting loops on a bare server, the problem isn’t a missing penalty (llama.cpp server README, smcleod sampling guide).

The sane starting recipe, in order of what to try:

1. Lower temperature. For factual/coding work use 0.1–0.3. High temperature on a small model is a common loop trigger.
2. A light repeat penalty, if any. Start at 1.05. Never exceed 1.2 — beyond that you distort the distribution and cause the exact looping you’re trying to stop (smcleod sampling guide).
3. Set repeat_last_n to 64 for prose, 128–512 for code (so it doesn’t penalize repeated variable names).
4. Turn on DRY — it targets phrase repetition far more precisely than a flat token penalty:

llama-server -m model.gguf \
  --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --dry-penalty-last-n -1

If loops persist, raise the multiplier to 1.0–1.5; if output goes incoherent, back off.

5. Fix sampler order. The default llama.cpp order applies penalties first, which interacts badly with some quantized models. Daniel Han’s recommended order for stubborn models:

--samplers "top_k;top_p;min_p;temperature;dry;typ_p;xtc"

In Ollama, set these in the Modelfile or per-request:

PARAMETER temperature 0.2
PARAMETER repeat_penalty 1.05
PARAMETER repeat_last_n 64

Some models ship their own guidance — the Unsloth GLM-4.7-Flash GGUF, for instance, explicitly recommends disabling repeat penalty (set to 1.0) and relying on DRY, because the penalty was pushing it into loops. Always check the model card before assuming a universal setting.

One trap worth naming: constrained JSON generation. gemma4:31b enters word-repetition loops inside JSON string fields regardless of penalty — 1.0, 1.15, and 1.5 all fail identically — because the grammar constraint permits any string content, so the sampler can’t reject a degenerate token (Ollama issue #15502). If you’re seeing loops only under format=json or a JSON schema, that’s the cause; relax the constraint or post-process instead.

Fix 3: GGGGGG and pure garbage from token one

If the model emits GGGGGGGGGGGG, rows of !!!!!, or symbol soup from the very first token, sampling won’t save you — the problem is upstream. This is a known llama.cpp behavior on the Vulkan backend: Qwen3-8B (Q4_K_M), Qwen2.5-7B, and Mistral-7B-Instruct-v0.1 all produced GGGGGG loops on an AMD Radeon RX 5700 (8 GB), Windows 10, LM Studio v0.3.15, while Gemma models on the same setup were fine (llama.cpp issue #13310).

Work through these in order:

• Switch the backend. In LM Studio, if you’re on Vulkan and hitting GGGG, try the ROCm or CPU runtime (Settings → Runtime). On NVIDIA, make sure you’re on the CUDA build, not a fallback. A backend swap fixes this more often than anything else.
• Re-download the quant. A truncated or corrupt GGUF (interrupted download, bad mirror) produces garbage. Verify the file size against the Hugging Face listing and re-pull if it’s short.
• Try a different quant level. Some aggressive quants (Q2_K, IQ-series) degrade badly on certain models. Step up to Q4_K_M or Q5_K_M as a test — if the garbage clears, the low-bit quant was the problem. Our quantization explainer covers which levels are safe.
• Check hardware support. Older or unusual GPUs (like the RX 5700 above) hit code paths that newer cards don’t. If a CPU-only run is clean but the GPU run is garbage, it’s a backend/driver issue, not the file.

Fix 4: Context overflow repetition

If the model runs clean for a while then degrades into loops on long chats or long documents, you’ve likely overflowed the context window. Once the prompt exceeds num_ctx (Ollama) or -c (llama.cpp), the runner truncates, the conversation state gets mangled, and the model starts repeating.

Set a max output ceiling so a degenerating model can’t fill your screen with repeated garbage, and raise context deliberately rather than leaving it at a tiny default:

# Ollama: raise context for long inputs
$ ollama run qwen3.6 --num-ctx 8192

# llama.cpp
llama-server -m model.gguf -c 8192

Bigger context costs VRAM, so if you get an OOM after bumping it, see our guide on fixing CUDA out of memory for KV-cache quantization and flash attention.

The Ollama bug that makes your repeat penalty do nothing

Here’s a 2026-specific gotcha that will waste an afternoon if you don’t know about it. On the newer ollamarunner path — used by Gemma 4 and other recent models — Ollama’s Go-native sampler in sample/samplers.go silently ignores repeat_penalty, frequency_penalty, and presence_penalty. The NewSampler() function only wires up temperature, top_k, top_p, min_p, seed, and grammar; the penalty options from your request are simply never passed through (Ollama issue #15783, opened Apr 24 2026).

The most visible fallout is audio transcription, where the missing repeat penalty drives 84–93% word error rate on longer utterances due to repetition loops. But it affects any newer model: you set repeat_penalty 1.3 in your Modelfile, restart, and nothing changes — because the value is discarded.

What to do about it:

• Don’t rely on repeat_penalty to rescue a newer model on Ollama. Control repetition with temperature and min_p (which are implemented), and fix the root cause (usually the template).
• Models on the older llamarunner path still honor penalties correctly — that path delegates to llama.cpp’s C++ sampler.
• For serious sampling control, run the model directly in llama.cpp (llama-server), where DRY, sampler order, and all penalties work as documented.
• Keep Ollama current — the latest stable is v0.30.10 (Jun 17 2026) — and watch that issue for a fix.

If you build agentic or coding workflows on top of a local model and hit inconsistent output, aicoderscope.com covers the tooling side (Continue, Cline, Aider) where these sampling quirks bite hardest.

Quick reference: settings that stop repetition

Setting	Safe value	When to change it
temperature	0.1–0.3 (factual), 0.7 (creative)	Lower first when looping
repeat_penalty	1.0–1.05	Never above 1.2; ignored on Ollama’s new runner
repeat_last_n	64 (prose), 128–512 (code)	Raise for code to spare variable names
DRY multiplier	0.8	Raise to 1.0–1.5 for stubborn loops
stop token	model-specific	Always set for small models
num_ctx / -c	8192+	Raise for long docs/chats

FAQ

Why does my model repeat only after a long conversation? Context overflow. The prompt exceeded the context window, the runner truncated it, and the state got corrupted. Raise num_ctx/-c, or start a fresh chat.

I set repeat_penalty in Ollama and nothing changed. Why? If you’re on Gemma 4 or another newer model, Ollama’s new Go sampler ignores it entirely (issue #15783). Use temperature and min_p, or run the model in llama.cpp.

Is a higher repeat penalty always better for stopping loops? No — the opposite. Above ~1.2 it distorts the token distribution and causes looping. Use DRY and lower temperature instead.

My model prints <start_of_turn> or <|im_end|> in its replies. What’s wrong? Wrong chat template. Re-pull from the official Ollama tag, or override with --chat-template-file in llama.cpp. See Fix 1.

Why GGGGGG specifically? It’s a backend/quant failure, not sampling — commonly seen on the Vulkan backend with certain models and older AMD cards. Switch backends or re-download the quant (issue #13310).

Does quantization level cause gibberish? It can. Very low-bit quants (Q2_K, some IQ variants) degrade badly on some models. Test with Q4_K_M or Q5_K_M; if it clears up, the quant was too aggressive.

Sources

• Go sampler (ollamarunner) silently ignores repeat_penalty/frequency_penalty/presence_penalty — Ollama issue #15783
• gemma4:31b repetition loop during constrained JSON generation — Ollama issue #15502
• GGUF imported models spit out gibberish — Ollama issue #2334
• Qwen3-8B and others output GGGGGG on Vulkan backend — llama.cpp issue #13310
• LLM Sampling Parameters Guide — smcleod.net
• QwQ-32B: How to Run Effectively (DRY, sampler order) — Unsloth
• Saving to GGUF & chat templates — Unsloth Documentation
• Gemma 4 GGUF Chat Template Fix — OpenAIToolsHub
• llama.cpp server README (sampling defaults) — GitHub

Last updated July 1, 2026. Ollama v0.30.10 current. Sampler behavior and bug status change between releases; verify against your installed version with ollama --version.