TL;DR: DFlash2 is a 2B block-diffusion draft model (released August 18, 2026, Apache 2.0) that speeds up Qwen3.8-27B losslessly. The 3.43× headline is real but measured on SGLang/vLLM at concurrency 1 — the llama.cpp implementation that merged August 27 measures 1.81×, and an RTX 3090 running vLLM lands at roughly 2.6×. You need a 24GB card and one extra --spec-type flag; Ollama can’t do it yet.

What you’ll be able to do after this guide:

• Run Qwen3.8-27B with the DFlash2 drafter in llama.cpp using the correct September 2026 flags — and know which GGUF repo to avoid.
• Budget the VRAM honestly: target + draft + KV cache on a 24GB card at 32K context.
• Dodge the three known traps: the broken incoai GGUFs, the Vulkan wrong-output bug, and the vision crash that got closed as “not planned.”

Honest take: If your Qwen3.8-27B already runs with MTP speculative decoding, DFlash2 buys you a few percent, not a revolution — llama.cpp’s own numbers put them nearly even. If you’re running plain autoregressive decode, this is the biggest free speed bump you can get for 1–2GB of VRAM. Either way, ignore the 3.43× number for home use: plan around 1.8–2.7×.

What DFlash2 actually is (and isn’t)

Qwen3.8-27B has been the default 24GB-card model since its weights dropped on August 14, 2026. On August 18, Inco AI and the z-lab research team released DFlash 2, a purpose-trained draft model that makes it decode faster without changing a single output token.

It is not a smaller Qwen you chat with. DFlash2 is a ~2B-parameter block-diffusion drafter: instead of guessing one token at a time the way a classic draft model does, it predicts a whole block of tokens in a single pass, keeps the top candidates at every position, and runs a lightweight selector that traces one coherent path through them. DFlash 2 specifically adds grouped dynamic depthwise convolution and that candidate selector on top of the original DFlash design. Qwen3.8-27B then verifies the drafted block in one forward pass — every token it accepts is a token you got at draft-model cost. Rejected drafts fall back to normal decoding, which is why the output is bit-for-bit identical to running the 27B alone.

If speculative decoding itself is new to you, our speculative decoding setup guide covers the mechanics and the 2026 flag renames in llama.cpp; the deeper theory is in why local LLMs got good in 2026.

The license is Apache 2.0, same as the target model, so there’s no fine print to check before commercial use.

The 3.43× claim vs what you’ll actually measure

Every post about DFlash2 quotes 3.43×. The number is real — and it comes from serving engines under conditions that don’t match a home rig. Here is what’s actually on record, engine by engine:

Setup	Baseline	With DFlash2	Speedup	Source
SGLang/vLLM, concurrency 1	—	—	3.11–3.43× (task-dependent)	z-lab model card
SGLang/vLLM, concurrency 32	—	—	1.01–1.45×	z-lab model card
llama.cpp (merged PR), M5 Pro 64GB, Q4_K_M	10.42 tok/s	18.89 tok/s	1.81×	PR #27342
vLLM 0.28.0, RTX 3090, W4A16	46 tok/s	118–126 tok/s	~2.6–2.7×	syv-ai benchmark repo
vLLM 0.28.0, RTX 3090, MTP instead	46 tok/s	114–124 tok/s	~2.5–2.7×	syv-ai benchmark repo

Three things jump out of that table:

The speedup collapses under concurrency. At 32 parallel requests the gain is 1.01–1.45×, because the GPU is already compute-bound and there’s no idle bandwidth for the drafter to exploit. Speculative decoding is a single-user optimization. That’s fine — a home lab is a single user — but it means the 3.43× ceiling belongs to exactly one request hitting an otherwise idle serving engine.

llama.cpp’s own measurement is 1.81×. The PR that merged DFlash2 support (August 27, 2026) reports Qwen3.8-27B Q4_K_M going from 10.42 to 18.89 tok/s on an Apple M5 Pro, with 5.03 drafted tokens accepted per rejection. Our site’s canonical RTX 3090 number for this model in llama.cpp is about 41 tok/s — if the 1.81× ratio holds on CUDA, that’s roughly 74 tok/s, which is an estimate, not a measurement. Nobody has published a clean 3090 llama.cpp DFlash2 number yet.

DFlash2 barely beats MTP. Qwen3.8-27B ships with multi-token-prediction heads that llama.cpp and vLLM already use as a built-in drafter. On the RTX 3090 vLLM benchmark, DFlash2 manages 118–126 tok/s against MTP’s 114–124 — an Ollama maintainer reviewing the DFlash2 port put it bluntly: “DFlash2 and MTP look about the same within a few percent overall.” DFlash2’s edge is acceptance depth (3.14–3.34 tokens per step vs MTP’s 2.8–2.9), which compounds on predictable output like code.

VRAM budget: this is a 24GB-card feature

The draft model is small, but it stacks on top of a target that already crowds 16GB cards out:

Component	Size
Qwen3.8-27B Q4_K_M (target)	16.8 GB
DFlash2 draft, Q4_K_M	1.14 GB
DFlash2 draft, Q8_0	2.06 GB
DFlash2 draft, BF16	3.86 GB

Target plus Q8_0 draft is ~18.9GB of weights. On a 24GB card that leaves about 5GB for context — arithmetic, not a measurement, but comfortable for 32K context with quantized (q8_0) KV cache, since the drafter’s own KV footprint stays small thanks to its sliding-window attention. On a 16GB card the target alone already spills to system RAM, and a spilled target makes speculative decoding pointless — you’d be accelerating a model that’s now bottlenecked on PCIe. The drafter is worth quantizing less than the target: at 2GB, Q8_0 costs you almost nothing and keeps draft quality (and therefore acceptance rate) up.

Practically that means a used RTX 3090 is the entry ticket, same as for the base model — our 24GB VRAM model guide covers what else that card unlocks, and the VRAM calculator will check your exact model + context combination. No card and no plans to buy one? A cloud 3090/4090 on RunPod runs this identical setup, and speculative decoding cuts your billed wall-clock time the same way it cuts your local waits.

llama.cpp setup that works (September 2026)

You need a llama.cpp build from August 27, 2026 or newer — DFlash2 support merged in PR #27342 and is auto-detected from the GGUF’s tensors, so there’s no new --spec-type value to learn. Update first; on an older build the draft loads as a plain (wrong) draft model or fails outright.

Download both models from the z-lab repos and start the server:

./llama-server \
  -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
  -hfd z-lab/Qwen3.8-27B-DFlash2-GGUF:Q8_0 \
  --spec-type draft-dflash \
  --spec-draft-n-max 7 \
  -ngl 99 -ngld 99 \
  -c 32768 --cache-type-k q8_0 --cache-type-v q8_0

What each piece does:

• -hfd pulls the draft GGUF from Hugging Face (use -md path/to/file.gguf for a local file).
• --spec-type draft-dflash selects the DFlash path; the build detects DFlash 2 checkpoints automatically from the new GGUF architecture constants.
• --spec-draft-n-max 7 is the draft depth the upstream example uses. The PR tested 4–8; start at 7 and tune. Code and structured output tolerate higher depths, creative prose rejects more and wants lower.
• -ngld 99 keeps every draft layer on the GPU. A drafter that spills to CPU defeats the purpose.

If it’s working, the server log prints speculative statistics as it generates — the merged PR’s reference run shows the shape of a healthy result: 5.03 accepted tokens per rejection and decode throughput up 1.81× over the same command without the draft flags. Quick sanity check: run a prompt with and without the -hfd/--spec-type pair and compare the reported tokens per second. If the two numbers are within noise of each other, your acceptance rate is bad or the drafter isn’t engaged — check the log for the draft load line.

One flag pair worth knowing from the general speculative decoding guide: the old --draft-max/--draft-min flags were renamed to --spec-draft-n-max/--spec-draft-n-min in 2026. Guides written before the rename will hand you commands that error out.

Ollama can’t do this yet — here’s the workaround

As of Ollama 0.32.x there is no DFlash2 support. The gap is documented in public issues: a user on Ollama 0.32.9 reported in August 2026 that even DFlash v1 “does not do anything” on their machine (issue #17683), and the actual DFlash2 port (PR #17865) is still open, targets the Apple-only MLX backend, and — per its own benchmarks — lands at 43.67 tok/s on an M5 Pro, about 16% faster than Ollama’s existing MTP path. There is no qwen3.8-27b-dflash2 tag in the Ollama library to pull.

The workaround is the standard one: run llama-server with the command above and point your usual front end at it. llama-server speaks the OpenAI-compatible API on port 8080, so Open WebUI, Continue.dev, or Cline connect to http://localhost:8080/v1 exactly as they would to Ollama. You lose Ollama’s model management, you keep your entire UI. (Using the accelerated model as a coding backend is its best use case — drafted tokens hit hardest on predictable output like code, and our sister site covers wiring local models into coding agents in depth. For the FOSS self-hosting angle on the whole stack, see aifoss.dev.)

The three traps (all verified, all current)

1. The incoai GGUFs are broken. The llama.cpp PR notes state it directly: GGUF conversions from the incoai account are known-broken, and any DFlash2 GGUF generated before August 27, 2026 needs reconversion. Download from z-lab/Qwen3.8-27B-DFlash2-GGUF — the official repo, converted after the format settled. If you grabbed a draft GGUF in the first week and it loads but produces no speedup, this is why.

2. Vision input crashes the draft pipeline. Qwen3.8-27B is multimodal; DFlash2’s draft path is not. Feed the server an image with the drafter attached and you get:

init: the tokens of sequence 0 in the input batch have inconsistent
sequence positions ... it is required that the sequence positions
remain consecutive: Y = X + 1

Issue #27862 tracking this was closed “not planned” in late August — text-only is the supported mode for now. The fix is operational, not technical: run a second, draft-free llama-server instance (different port) for vision requests, or drop the -hfd/--spec-type flags when you need images. Don’t wait for a patch.

3. Vulkan gives wrong output on older builds. The Vulkan backend produced incorrect DFlash2 output until a graph-optimizer fix (PR #27812); multi-GPU tensor split (-sm tensor) was also initially unsupported and fixed separately in PR #27858. Both fixes are in current builds — one more reason “update llama.cpp first” is step zero. CUDA users were never affected.

When to skip DFlash2 entirely

• You serve multiple users. Past a handful of concurrent requests the gain decays toward 1.0× (1.01–1.45× at concurrency 32). Batch throughput on a home card is already covered by vLLM’s continuous batching — the 3090 does ~1,035 tok/s aggregate at 64 concurrent without any drafter.
• Your VRAM is already at the edge. If adding 1–2GB of drafter forces the target’s layers or KV cache off the GPU, you’ll lose more than you gain. This is the same trade documented for classic draft models, and CUDA out-of-memory fixes won’t save a setup that’s fundamentally 2GB too big.
• You mostly generate high-entropy creative text. Acceptance rate is the whole game, and prose rejects drafts far more than code. z-lab’s own 3.11–3.43× spread is task-dependent for exactly this reason — the low end of every published DFlash2 range is the creative-writing end.
• You’re happy with MTP. It’s already on, it costs no extra VRAM, and it’s within a few percent of DFlash2 for most workloads. DFlash2 is for squeezing the last drops out of a dedicated single-user box.

FAQ

Does DFlash2 change the model’s output quality? No. Speculative decoding is lossless by construction — the target model verifies every drafted token, and rejected drafts are regenerated normally. The z-lab card states the output distribution is preserved exactly.

Do the draft and target quantizations have to match? No — they’re separate models with separate files. The upstream example pairs Q4_K_M with Q4_K_M, but the drafter is small enough that Q8_0 (2.06GB) is the better default: higher draft fidelity means higher acceptance, and acceptance is what pays for everything.

Will this work with the fine-tuned Qwen3.8-27B I trained? A LoRA-merged fine-tune of the same base usually keeps enough of the output distribution for the drafter to stay useful, but acceptance rate will drop with distance from the base model. Test with and without, and see our Unsloth fine-tuning guide for the training side.

Is there a DFlash2 for other models? z-lab has published drafters per target — alongside the Qwen3.8-27B version there’s a Muse Glimmer 30B variant (~3B drafter). Each target model needs its own trained draft; you cannot point the Qwen drafter at a different family.

When will Ollama support it natively? Open question. The MLX port (PR #17865) is Apple-only and unmerged, and a maintainer’s review noted DFlash2 and MTP perform about the same — which reads like low urgency. Watch that PR rather than release notes.

Sources

• z-lab/Qwen3.8-27B-DFlash2 — Hugging Face model card
• z-lab/Qwen3.8-27B-DFlash2-GGUF — official GGUF conversions
• DFlash 2: Keep Drafting Parallel — Inco AI announcement
• spec: add DFlash2 support — llama.cpp PR #27342 (merged Aug 27, 2026)
• Eval bug: DFlash2 vision support is still broken — llama.cpp issue #27862
• DFlash does not seem to be working — Ollama issue #17683
• DFlash2 MLX support — Ollama PR #17865
• Qwen3.8-27B on a single RTX 3090 with vLLM — syv-ai benchmarks
• What Is DFlash 2? Speculative Decoding Explained — MindStudio

Last updated September 7, 2026. Software versions and flags change fast — check the llama.cpp release notes if a command errors out.