Jul 24, 2026

RDNA4 Vulkan vs ROCm 7.2 for Local LLMs: Which Backend Is Faster on the RX 9000 Series in 2026?

By RunAIHome Team · 13 min read

amdrdna4vulkanrocmllama-cppollamalocal-llmgpu

TL;DR: On RDNA4, Vulkan is no longer the slow fallback — it beats ROCm by ~35% on small-model and MoE decode in llama.cpp, while ROCm wins large dense models (27B: 42.8 vs 29.1 tok/s on the R9700) and prompt processing. Pick the backend per workload, not per ideology.

	Vulkan (RADV)	ROCm 7.2 (HIP)	Both installed
Best for	7B–14B models, MoE (Qwen3.5-35B-A3B class), quick setup, any distro	Dense 20B+ models, vLLM/PyTorch, max prefill	Benchmarking your exact model, switching per job
Setup cost	Mesa drivers you already have	~30GB ROCm stack, Ubuntu-first	Disk space + one cmake flag each
The catch	Loses to ROCm on 27B dense decode by ~47%	Loses 7B Q4_0 decode by ~35 tok/s; vLLM FP8 still broken on gfx1201	Two llama.cpp builds to maintain

Honest take: Start with Vulkan — it’s already installed, and on RDNA4 it wins the models most people actually run. Add ROCm only when a dense 20B+ model or vLLM/PyTorch forces your hand.

For years the AMD advice was simple: suffer through the ROCm install because Vulkan was the compatibility fallback that cost you 30–50% of your tokens. On RDNA4 that advice is dead. The RX 9070 XT decodes Llama 2 7B Q4_0 at 137 tok/s on Vulkan vs 101 tok/s on ROCm in llama.cpp’s community scoreboards — the fallback is now the fast path for half the workloads that matter.

But only half. The same scoreboards show ROCm winning dense 27B decode by 47% on the Radeon AI PRO R9700. Which backend is “faster” depends on the shape of the model you run, and this article gives you the actual numbers, the build commands that changed in 2026, and the Ollama switch that picks between them.

What ROCm 7.2 changed for RDNA4

ROCm 7.2 (January 2026) supports RDNA4 natively: gfx1201 (Navi 48 — RX 9070, RX 9070 XT, Radeon AI PRO R9700) and gfx1200 (Navi 44 — RX 9060 XT) are officially listed in AMD’s compatibility matrix, with the RX 9060 tier added back in ROCm 7.0.2 per Phoronix. Native means exactly one thing worth money: no HSA_OVERRIDE_GFX_VERSION workaround. If you came from RDNA2/RDNA3 and reflexively set that variable, don’t — on a natively supported card it can only hurt. (If rocminfo doesn’t show your card at all, that’s a different problem: see our AMD GPU not detected fix.)

Check that ROCm actually sees the card before benchmarking anything:

$ rocminfo | grep gfx
  Name:                    gfx1201

If you see gfx1201 (or gfx1200 on a 9060 XT), the HIP backend will build and run without overrides. On Windows, the ROCm situation is narrower — only gfx1200/gfx1201 are supported at all, which we covered in our ROCm on Windows test.

Vulkan needs none of this. Any distro with Mesa’s RADV driver — which is every mainstream Linux desktop since 2020 — runs llama.cpp’s Vulkan backend out of the box. That asymmetry in setup cost is why the benchmark numbers below matter so much: ROCm has to win to justify its ~30GB install.

The head-to-head numbers

llama.cpp maintains standardized community scoreboards for both backends — same test (Llama 2 7B Q4_0, pp512 prompt processing, tg128 token generation) across hundreds of GPUs: the Vulkan scoreboard and the ROCm/HIP scoreboard. Here’s every RDNA4 card that appears in both, best submitted result per card:

GPU	Vulkan decode (tg128)	ROCm decode (tg128)	Vulkan prefill (pp512)	ROCm prefill (pp512)
RX 9070 XT	137.1 tok/s	101.3 tok/s	5,036 tok/s	5,055 tok/s
AI PRO R9700	138.8 tok/s (FA)	98.0 tok/s (FA)	5,620 tok/s (FA)	4,773 tok/s (FA)
RX 9070	119.7 tok/s	114.5 tok/s	3,164 tok/s	2,382 tok/s
RX 9060 XT	70.5 tok/s	67.6 tok/s	2,142 tok/s	1,480 tok/s (FA)

Llama 2 7B Q4_0, llama-bench, community-submitted results on different rigs and llama.cpp builds (mid-2026 commits) — treat deltas under ~10% as noise. FA = flash attention enabled where it was the faster submission.

Two things jump out. First, on the two 64-CU cards (9070 XT, R9700) Vulkan decodes ~35–42% faster than ROCm on this test — that’s not noise, that’s a different kernel quality tier. Second, prefill is close to a wash on the big cards, and Vulkan actually wins it on the smaller ones.

The back-of-envelope bandwidth math (our calculation, not a benchmark): a 7B Q4_0 GGUF is ~3.8GB of weights, so 137 tok/s means reading ~520 GB/s — about 81% of the RX 9070 XT’s 640 GB/s rated bandwidth (16GB GDDR6, 256-bit at 20 Gbps, 304W). ROCm’s 101 tok/s works out to ~60% utilization. Decode is bandwidth-bound, so that gap is pure software: RADV’s compute shaders are simply extracting more of the memory system than AMD’s own HIP kernels on this workload.

The plot twist: model shape flips the winner

If the story ended there you’d just uninstall ROCm. It doesn’t. The RDNA4 tuning thread on the R9700 — same silicon class as the 9070 XT with 32GB — found the split that the 7B scoreboard hides:

Dense 27B models: ROCm wins big. Qwen3.6-27B decode hit 42.8 tok/s on ROCm vs 29.1 tok/s on Vulkan (Q8-class quant with quantized KV) — a 47% ROCm advantage, with ROCm prefill around 960 tok/s on a 3.7K-token prompt.
MoE models with small active parameter counts: Vulkan wins. Qwen3.5-35B-A3B (~3B active per token) decoded at 127 tok/s stock on Vulkan, 156 tok/s tuned — the thread’s conclusion is that Vulkan leads on MoE under ~20B active while ROCm leads on 20B+ dense.

The pattern is consistent with the 7B result: Vulkan’s kernels are strongest exactly where decode looks like “small active weight set, many tokens” — 7B–14B dense and every consumer-relevant MoE. ROCm’s kernels (and its rocWMMA flash-attention path) pull ahead when each token has to chew through 15GB+ of dense weights.

So the practical rule for an RDNA4 owner in 2026:

Qwen3.5/3.6 MoE, Gemma 4 26B-A4B, anything 7B–14B → Vulkan.
Dense 22B–32B (Codestral 2, Qwen3.6-27B, Gemma 4 31B) → ROCm.
Long prompts dominate your workload (RAG, agent loops) → benchmark both; prefill is closer, and ROCm’s advantage grows with dense model size.

Building llama.cpp both ways

A warning first, because half the tutorials in the SERP still get this wrong: the flag is -DGGML_HIP=ON, not the older -DGGML_ROCM you’ll see in stale guides, and the target selector is -DGPU_TARGETS per the current llama.cpp build docs. (The separate Kompute backend some 2024-era posts mention is gone — Vulkan is its own first-class backend.)

Vulkan build (any distro, ~2 minutes of dependencies):

sudo apt install libvulkan-dev glslc spirv-headers
cmake -B build-vulkan -DGGML_VULKAN=ON
cmake --build build-vulkan --config Release -j

ROCm/HIP build (after installing ROCm 7.2 — full walkthrough in our ROCm Ubuntu setup guide):

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
  cmake -B build-rocm -DGGML_HIP=ON -DGPU_TARGETS=gfx1201 \
  -DGGML_HIP_ROCWMMA_FATTN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build-rocm --config Release -j

Use gfx1200 on an RX 9060 XT. -DGGML_HIP_ROCWMMA_FATTN=ON enables the rocWMMA flash-attention path the docs recommend for RDNA3+ — it’s part of why ROCm’s dense-model numbers improved this year, and pre-ROCm-7 builds couldn’t use WMMA on RDNA4 at all.

Then benchmark your model with both binaries — this is the command that settles every backend argument in 90 seconds:

$ ./build-vulkan/bin/llama-bench -m qwen3.6-27b-Q4_K_M.gguf -fa 1
| model            | backend | test  |            t/s |
| ---------------- | ------- | ----- | -------------: |
| qwen3 27B Q4_K_M | Vulkan  | pp512 |  798.42 ± 1.31 |
| qwen3 27B Q4_K_M | Vulkan  | tg128 |   29.07 ± 0.02 |

Run the same line against build-rocm/bin/llama-bench and keep whichever wins. Two builds, one model file — GGUFs work identically on both.

Squeezing more from Vulkan

The R9700 tuning thread’s flags apply to the whole RX 9000 series:

-ub 2048 -b 16384 — +29% prefill on MoE models.
GGML_VK_ALLOW_GRAPHICS_QUEUE=1 — +5% MoE decode, but –8% on dense; it’s workload-specific, test it.
Newer Mesa helps more than any flag: the thread measured double-digit MoE gains just moving to Mesa 25.3+.

Ollama: one variable picks the backend

Ollama ships its own bundled runtimes, and here’s the real-world failure that makes backend choice concrete. When the R9700 launched, Ollama’s bundled ROCm 6.x had no gfx1201 kernels — the server log showed failed to finish discovery before timeout then total vram = 0 B, and inference silently fell back to CPU (Ollama issue #13236). The card worked; the bundled backend didn’t know it existed. The fix was one line on the service:

sudo systemctl edit ollama
# add:
[Service]
Environment="OLLAMA_VULKAN=1"

OLLAMA_VULKAN=1 enables the experimental Vulkan backend Ollama introduced in v0.12.6 and shipped in binary releases from v0.12.11. Set it where the service reads it (systemd drop-in, not your shell — the same trap as every other Ollama variable, see our Ollama GPU fix guide). OLLAMA_VULKAN=0 disables it again, and GGML_VK_VISIBLE_DEVICES selects specific Vulkan devices on multi-GPU or iGPU+dGPU systems. There is no OLLAMA_COMPUTE_BACKEND variable, whatever a chatbot told you — Vulkan-on/Vulkan-off is the whole interface. Verify with ollama ps after loading a model: you want 100% GPU in the PROCESSOR column.

On an RX 9070 XT running mostly 7B–14B and MoE models through Ollama, Vulkan is now arguably the better backend, not the workaround — the same ~35% decode edge from the scoreboards applies, since Ollama wraps the same llama.cpp kernels.

The vLLM asterisk: ROCm’s ecosystem is still ahead — when it works

Backend choice isn’t only llama.cpp. If you want vLLM, PyTorch fine-tuning, or ComfyUI’s ROCm path, Vulkan doesn’t play — HIP is the only door into the Python ML stack, and that alone justifies installing ROCm on a card you bought for AI.

Just know the door still sticks on RDNA4. A documented benchmark on the RX 9070 XT measured llama.cpp-Vulkan at 62 tok/s vs vLLM-on-ROCm at 48 tok/s on the same Qwen3.5-9B (different quants: Q6_K GGUF vs FP8, so read it as a stack comparison, not a kernel shootout). The root cause is that vLLM lacks gfx1201 platform detection and silently dequantizes every FP8 op to FP32, bypassing RDNA4’s 128 AI accelerators entirely (vllm-project/vllm#28649). Until that lands, llama.cpp is the performance path on RX 9000 cards, whichever backend you build it with.

Which backend, which card: the verdict

RX 9070 XT (~$670–710 street in July 2026 against a $599 MSRP, per TechPowerUp and current deal trackers): Vulkan first. At 16GB you’ll live in the 7B–14B dense and small-MoE zone where Vulkan wins decode outright. Install ROCm only for vLLM/PyTorch experiments.

RX 9060 XT (16GB): Vulkan, no contest — it wins decode and prefill on this card, and skipping the ROCm install on a budget build is pure profit.

Radeon AI PRO R9700 (32GB): the one genuine both-backends card. 32GB fits dense 27B–32B models where ROCm is 47% faster, and MoE models where Vulkan is. Keep two llama.cpp builds and switch per model — full numbers in our R9700 hardware guide.

And the honest cross-vendor note: none of this closes the gap with a used RTX 3090 (~$1,250, 936 GB/s), which decodes 7B-class models around 95–100 tok/s and gets CUDA’s entire ecosystem without backend gymnastics. RDNA4’s pitch is new-with-warranty pricing and rapidly improving software — a pitch that’s aging well, but on cadence, not on parity. If your workload outgrows a single card, RunPod rents A100s at $1.39/hr for the occasional big-model job while your 9070 XT handles the daily loop; and if you’re wiring the card into a coding assistant, our sister site covers local backends for Cursor and Cline where backend tok/s directly sets your autocomplete latency.

FAQ

Do I need ROCm installed to use the Vulkan backend? No. Vulkan needs only Mesa’s RADV driver (standard on every major distro) and the Vulkan SDK packages at build time. That independence from the ROCm stack — no kernel module, no 30GB install, no version matrix — is Vulkan’s core advantage.

Which gfx target is my card? RX 9070, RX 9070 XT, and AI PRO R9700 are gfx1201 (Navi 48); RX 9060 XT is gfx1200 (Navi 44). Confirm with rocminfo | grep gfx.

Does HSA_OVERRIDE_GFX_VERSION help on RDNA4? No — it’s for pointing unsupported cards at supported kernel binaries. RDNA4 is natively supported in ROCm 7.2; setting the override on gfx1200/gfx1201 is at best a no-op.

Why does Vulkan win 7B but lose 27B dense? Decode is memory-bandwidth-bound, and the two backends’ kernels extract different fractions of that bandwidth depending on weight-matrix shape. RADV’s compute shaders currently do better on small dense and MoE (small active set) shapes; ROCm’s WMMA/rocWMMA paths do better when each token streams 15GB+ of dense weights.

Can I use Vulkan on Windows? Yes — llama.cpp builds with -DGGML_VULKAN=ON on Windows, and it’s often the easier path there given Windows ROCm covers only gfx1200/gfx1201. LM Studio’s AMD builds also default to Vulkan runtimes.

Does Ollama’s Vulkan mode cost performance vs bare llama.cpp? Ollama wraps the same llama.cpp Vulkan kernels; expect the usual few-percent server overhead, same as its CUDA/ROCm paths. The backend gap (35%+ on the workloads above) dwarfs the wrapper overhead.

Recommended Gear

RX 9070 XT — 16GB, 640 GB/s, the RDNA4 sweet spot for 7B–14B and MoE models on Vulkan
RX 9060 XT 16GB — budget RDNA4 where Vulkan wins everything and you skip ROCm entirely
AMD Radeon AI PRO R9700 — 32GB, the one RDNA4 card worth maintaining both backends for

Sources

Last updated July 24, 2026. Prices and specs change; verify current rates before purchasing. Benchmark figures are community-submitted results on varying rigs and builds — run llama-bench on your own hardware before committing to a backend.

Was this article helpful?