Qwen3-Coder 480B-A35B Local Hardware Guide 2026: What GPU You Actually Need to Run the #1 Open-Weight Coding Model

By RunAIHome Team · 12 min read
qwen3-coderlocal-llmmoehardware-guidecoding-llmllama-cppquantizationrunpod

TL;DR: Qwen3-Coder 480B-A35B is the largest open-weight coding model in Qwen’s lineup — 480B total parameters, 35B active, and it trades punches with Claude Sonnet 4 on agentic coding benchmarks. But the smallest usable Unsloth quant is 150GB and a good one is 276GB, so no single consumer GPU touches it. Your three real options are the free/near-free API, a rented multi-GPU cloud pod, or a 150GB+ RAM local build that runs it slowly.

Free/cheap APIRunPod cloud GPULocal multi-GPU / big-RAM build
Best for99% of developersBursty heavy use, private-ishData-can’t-leave, always-on
Cost$0 (free tier) or ~$0.22/$1.80 per M tok~$5.60/hr for 4× A100 80GB~$10k–$13k hardware
Speed30–60+ tok/s (managed)30–50 tok/s~6–12 tok/s (Q2/CPU offload)
Memory neededNone320GB VRAM (rented)150–320GB RAM/VRAM
The catchPrompts leave your machineBilled by the hourSlow, expensive, loud

Honest take: For almost every home-lab developer, the correct answer is the free OpenRouter endpoint or the cheap Qwen API — this is a cloud-class model that happens to have open weights, not a card you buy. Build local only if your code genuinely cannot leave the building, and even then rent a RunPod pod first to confirm it’s worth it.

What Qwen3-Coder 480B-A35B actually is

Qwen3-Coder-480B-A35B-Instruct is a Mixture-of-Experts (MoE) model with 480 billion total parameters and 35 billion active per token — it activates 8 of 160 experts on each forward pass. Native context is 256K tokens, extendable to roughly 1M via YaRN. That combination is why it’s aimed squarely at agentic, repository-scale coding: reading a whole codebase, calling tools, and generating multi-file changes in one shot.

The benchmark numbers are the reason people want it locally. It scores 61.8% on Aider Polyglot and 66.5% Pass@1 on SWE-bench Verified — numbers that match or edge past Claude Sonnet 4 on agentic coding tasks. For an open-weight model you can download, inspect, and self-host, that’s the front of the pack.

The problem is the “self-host” part. 480 billion parameters do not care how good your single GPU is.

The VRAM reality: it’s a storage problem, not a speed problem

MoE routing means only 35B parameters are computed per token, but all 480B weights still have to live in memory — the router picks a different 8-of-160 experts on every token, so the whole set must be resident. That’s the wall.

Here are the real GGUF sizes from Unsloth’s dynamic quants, which are the community standard for this model:

Quant (Unsloth)File sizeQuality signalWhere it fits
UD-IQ1_M (~1-bit)~150GBLossy, usable for chat192GB unified mem or big-RAM CPU build
UD-Q2_K_XL (2-bit)~180GBSolid, common local pickMulti-GPU or 256GB DDR5 build
FP8~250GBNear-full precision4× H100 80GB
UD-Q4_K_XL (4-bit)~276GB60.9% Aider Polyglot (vs 61.8% bf16)4× A100 80GB / 8× H100
bf16 (full)~960GBReference12× H100 — not home territory

The standout is that the 276GB UD-Q4_K_XL scores 60.9% on Aider Polyglot — within a point of the full 960GB bf16 (61.8%). Unsloth’s dynamic 4-bit is the sweet spot: you get essentially frontier-open quality at under a third of the full model’s footprint. But 276GB is still four A100 80GB cards.

For comparison, a used RTX 3090 gives you 24GB of VRAM at 936 GB/s for around $1,050–$1,070 as of July 2026. To hold the 276GB Q4 quant purely in VRAM you’d need roughly twelve of them — about $12,000–$13,000 in GPUs alone, before the motherboard, PSUs, and PCIe lanes to feed them. This is why “which GPU runs Qwen3-Coder 480B” has an uncomfortable answer: none of the ones in your price range, alone.

Can you run it on a single consumer GPU? No — and here’s the math

A single RTX 5090 has 32GB of VRAM. A single RTX 4090 has 24GB. The smallest quant that produces coherent output (UD-IQ1_M) is 150GB. You are short by a factor of 5–6× on the best consumer card that exists.

The only single-box paths that hold the weights at all are:

  • A high-unified-memory machine — a 192GB+ Apple Silicon box or an AMD Ryzen AI Max+ 395 mini PC with 128GB (still short of the 150GB 1-bit quant). Note that Apple pulled its high-RAM Mac Studio configs in 2026, so new Apple hardware now tops out at 96GB unified — which can’t hold even the 1-bit file. See our Ryzen AI Max+ 395 write-up for what 128GB unified actually manages.
  • A big-RAM CPU/server build — 256GB+ DDR5, offloading the MoE layers to system RAM and keeping only the attention/non-expert layers on a single GPU.

Both are slow. Expect around 6+ tokens/second for the 2-bit quant on a 150–180GB unified-memory or CPU-offload setup — readable, but not the 30–60 tok/s you’d get from a managed endpoint. Prefill (reading a long prompt) is far worse on CPU-bound builds, which matters a lot for a 256K-context coding model where the whole point is feeding it big files.

The MoE CPU-offload trick (and the OOM it fixes)

If you do go the single-GPU-plus-RAM route with llama.cpp, the flag that makes it possible is expert offload. Instead of trying to cram all layers onto the GPU (which triggers an immediate ggml_backend_cuda_buffer_type_alloc: CUDA error: out of memory), you keep the dense layers on the GPU and push the fat MoE expert tensors to CPU RAM:

$ ./llama-cli \
    --model Qwen3-Coder-480B-A35B-UD-Q2_K_XL-00001-of-00004.gguf \
    -ngl 99 \
    -ot ".ffn_.*_exps.=CPU" \
    -c 32768 \
    -p "Refactor this function for readability"

# ... loading ...
llama_model_loader: offloading 62 repeating layers to GPU
llama_kv_cache_init: CUDA0 KV buffer size =  2560.00 MiB
prompt eval time = 18244 ms
eval time: ~7.2 tokens/second

That -ot ".ffn_.*_exps.=CPU" regex is doing the heavy lifting — it routes every expert feed-forward tensor to system memory while your GPU handles attention. Without it, a 24GB card OOMs before the first token. With it, a single RTX 3090 plus 256GB of DDR5 can serve the 2-bit quant at single-digit tokens/second.

Path 1: The free (or nearly free) API — the right answer for most people

Qwen3-Coder 480B is available on OpenRouter at $0.22 per million input tokens and $1.80 per million output tokens, and there is a free tier endpoint on OpenRouter as well. It’s also served through Alibaba’s own Qwen API. For the overwhelming majority of readers, this is where the story ends: you get frontier-open coding quality, 30–60+ tok/s, and no hardware bill.

Run the numbers against a local build. Say you’re a heavy user burning 10 million input and 2 million output tokens a month through an agentic coding tool. On OpenRouter’s paid rate that’s about $2.20 + $3.60 = ~$5.80/month. A local rig that runs the model at usable quality costs $10,000+ up front and still runs slower. The break-even is measured in years, and that’s before electricity. If your priority is cost, the API wins by a mile.

If you’re wiring this into an editor, our sister site aicoderscope.com covers the coding-tool side — how Qwen3-Coder stacks up against Claude and GPT inside Cursor, Cline, and Aider.

Path 2: RunPod cloud GPU — when you need the weights but not the ownership

If you want to run the actual open weights (for privacy-adjacent reasons, fine-tuning, or because your prompts include proprietary code you’d rather keep in a pod you control), renting beats buying until you’re running the model many hours a day.

RunPod’s current pricing, verified July 2026: A100 80GB PCIe at $1.39/hr (SXM at $1.49/hr), and H100 80GB PCIe at $2.89/hr. To hold the 276GB UD-Q4_K_XL quant you want roughly 320GB of VRAM:

  • 4× A100 80GB (320GB) ≈ $5.56/hr — comfortably fits Q4, ~30–50 tok/s
  • 4× H100 80GB (320GB) ≈ $11.56/hr — faster prefill, overkill for pure inference

At $5.56/hr, an 8-hour workday costs about $44. If you run it two hours a day, that’s roughly $220/month — more than the API, but you’re holding the raw weights. The moment your usage is bursty (spin up, hammer it, spin down), RunPod is dramatically cheaper than owning idle hardware. Spin up a pod here: runpod.io. For a fuller rent-vs-buy breakdown, see our cloud GPU pricing guide and the RunPod vs local GPU decision framework.

Path 3: The local build — for the specific person who needs it

There’s a real, narrow audience for local Qwen3-Coder 480B: teams whose code genuinely cannot leave the premises, or individuals running it enough hours that even cloud rental adds up. If that’s you, the honest options are:

  1. A 256GB+ DDR5 CPU/server build with one 24GB GPU for offload — runs UD-Q2_K_XL (~180GB) at ~6–8 tok/s. Cheapest local path, slowest prefill.
  2. A multi-GPU rig — enough 24–48GB cards to reach ~180–280GB of VRAM. Fastest local path, but you’re buying and powering 8–12 GPUs, which is a five-figure spend and a serious power/cooling problem.

Neither is pleasant. Both are for people who’ve already decided the tradeoff is worth it. If you’re on the fence, you don’t need this — run the smaller locally-native model instead (next section).

The model you probably should run locally instead

Here’s the pragmatic pivot. If you want a coding model that actually fits your hardware, Qwen3-Coder 480B is the wrong target — its smaller sibling is the right one. Qwen3-Coder-Next (the ~30B-A3B “Flash” class) runs on a single 24GB GPU at real speed, and for day-to-day autocomplete, refactors, and single-file work, most developers can’t tell the difference. Full details in our Qwen3-Coder-Next hardware guide.

The mental model: use the 480B through the API when you need its repository-scale reasoning, and keep a 30B-class coder on your own GPU for the fast, private, everyday loop. That’s the setup that actually makes sense for a home lab — the same conclusion we reached in the open-source LLM shootout.

Cost comparison: three ways to use Qwen3-Coder 480B

Free APIRunPod (4× A100)Local build (Q2, offload)
Upfront cost$0$0~$4,000–$13,000
Running cost$0–~$6/mo (free/light)~$5.56/hr~$0.10–$0.30/hr electricity
Speed30–60+ tok/s30–50 tok/s~6–12 tok/s
QualityFullFull (Q4/FP8)Reduced (1–2 bit)
PrivacyPrompts leaveYour rented podFully local
Setup effortMinutes~30 minDays

The pattern is clear: cost and convenience both point to the API; only hard privacy or extreme usage justify the rest. This is the same story we told for Kimi K2.7 and MiniMax M3 — trillion-scale and near-trillion-scale open weights are, functionally, cloud models with a source-available license.

FAQ

Can I run Qwen3-Coder 480B on an RTX 4090 or 5090? Not on its own. A 5090 has 32GB of VRAM; the smallest coherent quant is 150GB. You can serve it from a single GPU only by offloading the MoE experts to 150GB+ of system RAM with llama.cpp’s -ot flag, and even then you’re looking at single-digit tokens/second.

What’s the smallest usable quant? Unsloth’s UD-IQ1_M at ~150GB for chat, or UD-Q2_K_XL at ~180GB for better coding quality. The UD-Q4_K_XL at 276GB is the real sweet spot — it scores 60.9% on Aider Polyglot versus 61.8% for the full 960GB bf16 model.

Is it actually as good as Claude Sonnet 4? On agentic coding benchmarks, close. It scores 66.5% Pass@1 on SWE-bench Verified and 61.8% on Aider Polyglot, matching or slightly exceeding Sonnet 4 on those specific tests. Real-world agentic reliability still favors the frontier closed models for the hardest tasks, but the gap is small and the weights are open.

How much does the API cost? OpenRouter lists $0.22 per million input tokens and $1.80 per million output tokens, with a free-tier endpoint also available. Alibaba’s Qwen API serves it too.

Should I buy hardware for this? Almost certainly not. Use the API for the 480B, and if you want a local coding model, buy a 24GB GPU (a used RTX 3090 at ~$1,050) and run Qwen3-Coder-Next instead.

If you’re building the local coding rig that actually makes sense (a fast 24GB card for the smaller coder, using the API for the 480B):

  • RTX 3090 — 24GB, 936 GB/s, ~$1,050 used; the tokens-per-dollar king for local inference
  • RTX 4090 — 24GB, faster prefill if you can find one near MSRP
  • RTX 5090 — 32GB, the only consumer card past 24GB, for headroom on 30B-class coders
  • Cloud alternative: RunPod for renting the multi-GPU pods that actually fit the 480B

Sources

Prices and availability verified as of July 2026. Hardware and GPU-rental prices move weekly — confirm current rates before buying or renting.

Was this article helpful?