TL;DR: ORL is an open-source (MIT) training recipe plus inference engine from Together AI and five universities that makes diffusion language models decode in larger parallel blocks. On a single RTX 5090 it generates 2,788 tok/s on HumanEval with a 16B MoE model; in the paper’s own H100 comparison, the autoregressive baseline manages 240 tok/s where ORL does 1,298. The catch: the engine runs only on Hopper and Blackwell GPUs, so the 5090 is the only consumer card that qualifies, and there is no GGUF, Ollama, or llama.cpp path at all.

	RTX 5090 32GB	Used RTX 3090 / 4090 (24GB)	Rented 5090/H100
Best for	The only consumer card ORL supports (NVFP4)	Everything else local — but not ORL	Trying ORL before any purchase
Price / Cost	$3,822–$5,000 street (MSRP $1,999)	$1,456–$1,558 / $2,575–$3,010 used	Vast.ai 5090 from $0.25/hr
The catch	Research-grade, Linux-only, single-request stack	dlmrt has no Ampere/Ada kernels — locked out today	Setup still needs PyTorch 2.11 + CUDA 12.9+

Honest take: Don’t buy a $4,000 GPU for ORL — it’s a research stack with three supported models and no mainstream runtime. But if you already own or were already buying an RTX 5090, this is the fastest local text generation that exists right now, by a wide margin, and it costs you nothing to try.

The fastest local LLM numbers posted this month didn’t come from a bigger GPU or a new quantization trick. They came from ORL (“On-Policy Residual Learning for Diffusion Language Models”), an October 2026 release from researchers at Together AI, UIUC, UC Berkeley, UC Santa Cruz, Stanford, and UT Austin (arXiv 2610.20687) that has been making the rounds on r/LocalLLaMA. The headline: a diffusion language model generating 1,298 tok/s on one H100 where the autoregressive baseline manages 240, and — more relevant to this site — measured single-card RTX 5090 results in the 1,490–2,788 tok/s range.

Numbers like that break the mental model most of us use to size GPUs (bandwidth ÷ weights read per token stops being the binding constraint), so this guide does three things: explains what ORL actually is, checks whether the speed costs quality, and answers the question the queue exists to answer — what hardware you’d actually need, and whether that should change what you buy.

What ORL is — and what it isn’t

ORL is not a new foundation model. It’s a post-training method plus a purpose-built inference engine, applied to three existing open diffusion language models:

• QwenDLM-4B — a 4B dense diffusion model (yifanyu/QwenDLM-4B-ORL on Hugging Face)
• LLaDA2.1-mini — Ant Group’s 16B-parameter MoE diffusion model, Apache 2.0, released February 2026 (yifanyu/LLaDA2.1-mini-ORL, plus an NVFP4 variant)
• DiffusionGemma 26B-A4B — Google’s experimental text-diffusion Gemma, which we covered in our DiffusionGemma hardware guide (yifanyu/DiffusionGemma-26B-A4B-it-ORL, plus NVFP4)

A diffusion LLM doesn’t write one token at a time. It drafts a whole block and refines it over denoising steps, so each forward pass can commit several tokens at once. The problem has always been that, out of the box, these models only safely commit 2–8 tokens per pass before quality degrades. ORL’s contribution is training the model on its own residual trajectories — the gaps between its parallel drafts and its own sequential answers — so it learns to commit much larger chunks confidently. On QwenDLM-4B, tokens-per-forward-pass goes from 1.8–2.4 to 5.8–11.1 depending on the task, which the authors report as up to 6.2× more parallelism and about 5× end-to-end throughput. Remarkably cheap, too: the LLaDA2.1-mini recipe used 4.2M self-generated tokens and 36 H100-hours — a fine-tune, not a pre-train.

The code and recipe are MIT-licensed; the inference engine, dlmrt, is Apache 2.0. Both are genuinely open. That’s the good news. The hardware support list is the bad news, and we’ll get there.

Why 2,788 tok/s is physically plausible (and why it’s not an autoregressive number)

Flag this prominently, because the first reaction to these benchmarks is “that’s fake”: diffusion tok/s and autoregressive tok/s are not the same quantity. An autoregressive model reads its active weights from VRAM once per token, which is why decode speed is capped at memory bandwidth divided by weights read per token — the rule this site uses to sanity-check every benchmark. A block-diffusion model reads the weights once per denoising pass and commits many tokens per pass, so the per-token weight traffic is divided by the block parallelism.

Run the arithmetic on DiffusionGemma-26B-A4B + ORL on an RTX 5090 (1,792 GB/s): ~3.8B active parameters at NVFP4 is roughly 2.1 GB of weight reads per pass, and ORL’s measured ~28–30 tokens committed per pass amortizes that to ~70–75 MB per token. The bandwidth ceiling at that traffic level sits above 20,000 tok/s — the measured 1,490 tok/s is nowhere near it. Diffusion inference on consumer cards is latency- and compute-bound (dlmrt measures 15.41 ms per denoising round for DiffusionGemma on the 5090, 3.47 ms for LLaDA2.1-mini), not bandwidth-bound. That inversion matters for buyers: the spec that makes the 5090 king for autoregressive work (1,792 GB/s) is not what’s earning these numbers — the Blackwell FP4 tensor cores are.

The same amortization logic is why speculative decoding speeds up ordinary models — and the ORL paper benchmarks directly against it. Single H100, single request, 4B-class model: autoregressive 240 tok/s, EAGLE3 speculative decoding 353–405 tok/s, DFlash block drafting 573–746 tok/s, QwenDLM + ORL on dlmrt 1,298 tok/s. Speculative decoding guesses tokens and verifies them; a diffusion model trained with ORL just is the parallel generator, no draft model, no acceptance-rate lottery.

The measured numbers on hardware you can buy

All figures below are the ORL authors’ published dlmrt benchmarks (single request, October 2026). The RTX 5090 rows are the consumer story:

Model	GPU	Precision	GSM8K	MATH-500	HumanEval	MBPP
QwenDLM-4B + ORL	1× H100	BF16	987	701	1,609	1,222
LLaDA2.1-mini + ORL	1× H100	BF16	1,281	1,627	2,124	1,539
LLaDA2.1-mini + ORL	1× RTX 5090	NVFP4	1,648	2,081	2,788	2,053
DiffusionGemma + ORL	1× RTX 5090	NVFP4	1,540	1,408	1,490	1,465
DiffusionGemma + ORL	1× B200	NVFP4	4,214	3,805	3,949	3,898

All numbers in tok/s. Two things stand out. First, the single RTX 5090 beats the H100 on LLaDA2.1-mini — NVFP4 on Blackwell tensor cores beats BF16 on Hopper for this workload. Second, compare against our existing canon: stock DiffusionGemma through vLLM’s NVFP4 path ran about 700 tok/s on the same 5090 in our June guide. ORL plus dlmrt roughly doubles that, on the same silicon, with the same 18 GB NVFP4 footprint.

Does the speed cost quality?

The usual trade with parallel decoding is accuracy, so here are the authors’ before/after scores (accuracy % / tokens committed per forward pass):

Model	GSM8K	HumanEval
QwenDLM-4B → + ORL	90.4 → 92.4 (2.4 → 5.9 tok/pass)	82.9 → 89.6 (1.8 → 11.1 tok/pass)
LLaDA2.1-mini → + ORL	91.9 → 92.0 (5.6 → 8.5)	86.6 → 89.0 (11.3 → 13.1)
DiffusionGemma → + ORL	95.8 → 96.0 (26.0 → 30.0)	90.9 → 91.9 (24.5 → 27.9)

Accuracy holds or improves across the board — HumanEval on QwenDLM actually jumps 6.7 points while parallelism goes up 6×. The honest caveat: these are the method authors’ own evaluations on four academic benchmarks, run through their own pipeline (SGLang/vLLM for accuracy). No independent replication exists yet, and benchmark suites are exactly where a post-training recipe looks its best. Treat the quality parity as “credible, not yet confirmed.” Also remember the baseline: DiffusionGemma itself still scores below standard Gemma 4 on Google’s own published benchmarks — ORL doesn’t change what the underlying models are.

What GPU you need — the uncomfortable part

Here’s where the trending-post excitement meets the support matrix. dlmrt’s documented requirements, as of October 2026:

• GPUs: NVIDIA Hopper and Blackwell only. H100, B200/B300, and the RTX 5090 (for the NVFP4 checkpoints). Ampere and Ada do not appear in the support list — there are no kernels for them.
• Software: Linux, Python 3.10+, PyTorch 2.11, Triton 3.6, CUDA toolkit 12.9+ (13.0+ for B300). The engine compiles its CUDA extensions on first use.
• Formats: BF16, FP8 block-scaled projections, and NVFP4 experts on Blackwell FP4 tensor cores. No GGUF. No Ollama, no llama.cpp, no LM Studio.
• Serving model: dlmrt is a single-request engine — one prompt at a time, built to measure and serve interactive latency, not batch concurrency.

Spelled out per card:

• RTX 5090 (32GB, $3,822–$5,000 street): the one consumer card on the list. Both NVFP4 checkpoints fit — DiffusionGemma’s NVFP4 build is ~18 GB of weights, LLaDA2.1-mini is a 16B MoE whose NVFP4 weights are smaller still — and the numbers in the table above are the authors’ own measurements on this card.
• RTX 4090 (24GB used, $2,575–$3,010): locked out. This is a harder lockout than DiffusionGemma’s original “no native NVFP4” problem — with vLLM you could at least fall back to a heavier format on Ada; dlmrt ships no Ada path at all.
• RTX 3090 (24GB used, $1,456–$1,558): same story. The value king for llama.cpp workloads has no entry here.
• RTX 5080/5070 Ti (16GB): Blackwell, so the architecture qualifies, but the authors validate only the 5090 and publish no 16GB-card results. LLaDA2.1-mini’s NVFP4 weights would arithmetic-wise squeeze under 16 GB before KV cache, but nobody has demonstrated it — treat 16GB Blackwell as unverified, not supported.

If that support matrix sounds familiar, it should: it’s the same “Blackwell gets the future first” pattern we flagged in the DiffusionGemma guide and the 5090 vs 4090 comparison. Diffusion LLM inference is being built NVFP4-first, and NVFP4 is hardware-gated to RTX 50-series and newer.

Getting it running (RTX 5090 or a rented card)

The README’s recommended path sidesteps most dependency pain by starting from the SGLang container, which already carries the right PyTorch:

# Recommended: start from the SGLang image (has PyTorch 2.11 + CUDA deps)
docker run --gpus all -it lmsysorg/sglang:v0.5.13 bash

git clone https://github.com/Introspective-Diffusion/dlmrt
cd dlmrt && pip install --no-deps --break-system-packages -e .

Then inference is a few lines:

from dlmrt import LLM

llm = LLM("yifanyu/LLaDA2.1-mini-ORL-NVFP4", role="orl")
print(llm.generate("Write a function that merges two sorted lists."))

Expected behavior on first run: a long pause while dlmrt compiles its CUDA extensions (this is documented, not a hang), then the HF checkpoint download, then generation fast enough that the output appears effectively all at once — at 2,000+ tok/s a 500-token answer lands in about a quarter of a second after prefill.

The failure we can save you from: installing dlmrt into a generic environment and hitting the extension-compile step without a full CUDA toolkit. A driver alone isn’t enough — the first-use compilation needs CUDA 12.9+ toolkit, PyTorch 2.11, and Triton 3.6, and a mismatched PyTorch/Triton pair fails at build time, not install time. The fix is the one the authors themselves recommend: run inside lmsysorg/sglang:v0.5.13, where the dependency set is already correct, and install with --no-deps so pip doesn’t “upgrade” you into a broken combination. Windows users: dlmrt is Linux-only — WSL2 or a rented instance, there is no native path.

Should ORL change what you buy?

The queue asked for a verdict, so here it is: no — unless you were already buying Blackwell.

The case against buying for ORL: it’s a three-model research stack, weeks old, single-request, Linux-only, with no independent quality replication and no GGUF bridge to the tooling 90% of home labs actually run. The models themselves are a 4B, a 16B MoE, and an experimental 26B MoE that trails its own autoregressive sibling on quality. Nobody should convert $4,000 into a GPU to run that stack as a daily driver. If your workloads live in Ollama and llama.cpp on a 24GB card, nothing about this release makes your hardware worse — the DFlash2 speculative-decoding path remains the best free speedup on Ampere/Ada, and your card keeps its resale value either way.

The case for paying attention anyway: this is the second major release this year (after DiffusionGemma itself) where the interesting inference work shipped Blackwell-first and skipped older consumer cards entirely, and it’s the first time a single consumer GPU has posted interactive-use throughput that beats an H100. If diffusion LLMs keep absorbing research attention — and a 5× throughput jump from 36 GPU-hours of fine-tuning says they will — “NVFP4-capable” quietly becomes a real line item in GPU longevity, next to VRAM. For a coding-assistant backend specifically, 2,788 tok/s HumanEval throughput is the kind of speed that makes a local model feel like a hosted one in your editor.

What to actually buy

Prices as of October 2026, from the comparison above:

Your situation	The move	Price	Where
Already buying a top-end card for local AI — ORL is a bonus	RTX 5090 32GB, the only consumer card dlmrt supports	$3,822–$5,000	Check price
On a 24GB RTX 3090/4090	Buy nothing — enable DFlash2 speculative decoding instead	$0	Our setup guide
Want to feel 2,000+ tok/s before spending anything	Rent an RTX 5090 by the hour and run the install above	from $0.25/hr	Vast.ai

The middle row is the honest one for most readers. The rational play if you’re curious is the bottom row: an evening on a rented 5090 costs less than $2 and tells you more than any benchmark table.

FAQ

Is ORL a model I can download? You download ORL-tuned versions of existing models — QwenDLM-4B, LLaDA2.1-mini, and DiffusionGemma 26B-A4B, all on Hugging Face under the yifanyu namespace, including NVFP4 builds of the latter two. The recipe (MIT) and the dlmrt engine (Apache 2.0) are on GitHub.

Why can’t my RTX 4090 run it when it ran DiffusionGemma? DiffusionGemma through vLLM had a slow non-NVFP4 fallback for Ada. dlmrt, the only engine with ORL support, ships kernels for Hopper and Blackwell only — the 4090 isn’t slow here, it’s unsupported.

Are these tok/s numbers comparable to the Ollama numbers I know? Directionally yes (both are end-to-end single-request generation speed), but don’t sanity-check them against the bandwidth-ceiling formula — diffusion models commit ~6–30 tokens per weight read, so they operate far below the bandwidth limit and are compute-bound instead.

Will there be a GGUF / Ollama version? Nothing announced. llama.cpp has no block-diffusion execution path today, and dlmrt’s speed depends on custom kernels and (for consumer cards) Blackwell FP4 tensor cores, so a quality GGUF port is a research project, not a file conversion.

Is LLaDA2.1-mini + ORL actually useful, or just fast? It posts 92.0 GSM8K / 89.0 HumanEval in the authors’ evals — mid-tier 2026 open-model quality, delivered absurdly fast. For chat and coding-assistant loops it’s genuinely interesting; for frontier-quality reasoning you’d still reach for a bigger autoregressive model.

Sources

• ORL: On-Policy Residual Learning for Diffusion Language Models — GitHub (Introspective-Diffusion) (method, checkpoints, quality tables, H100 throughput comparison, MIT license, Oct 2026)
• dlmrt inference engine — GitHub (Introspective-Diffusion) (supported GPUs, software requirements, RTX 5090/B200/H100 benchmark table, install path)
• ORL paper — arXiv 2610.20687 (Together AI, UIUC, UC Berkeley, UC Santa Cruz, Stanford, UT Austin)
• LLaDA2.1-mini model card — Hugging Face (inclusionAI) (16B MoE, Apache 2.0, Feb 2026 release)
• ORL checkpoint collection — Hugging Face (yifanyu) (QwenDLM-4B-ORL, LLaDA2.1-mini-ORL ± NVFP4, DiffusionGemma-26B-A4B-it-ORL ± NVFP4)
• ResalePrices — RTX 3090 used market tracking ($1,456–$1,558, Oct 20 2026)
• vLLM — aifoss.dev review (the accuracy-eval serving stack referenced by the paper)
• Site price index: RTX 5090 $3,822–$5,000 (verified Sep 21 2026), used RTX 4090 $2,575–$3,010 (verified Oct 9 2026)

Recommended Gear

• RTX 5090 32GB — $3,822–$5,000; the only consumer GPU on dlmrt’s support list, and it outruns an H100 on LLaDA2.1-mini NVFP4
• Used RTX 4090 24GB — $2,575–$3,010; still the better buy for llama.cpp workloads, just not an ORL card
• Used RTX 3090 24GB — $1,456–$1,558; the 24GB value path if ORL isn’t your use case

Prices verified as of September 21 – October 20, 2026 (see Sources). GPU street prices move weekly; check current listings before buying.

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe