Used RTX 4070 for Local AI in 2026: The Fastest Cheap 12GB Card, While a New 5070 Costs $900

By RunAIHome Team · 12 min read
rtx-4070used-gpugpulocal-llmbuying-guide12gb-vram

TL;DR: A used RTX 4070 sells for $481–$519 in early September 2026, while the new RTX 5070 — the card that was supposed to replace it at $549 — now averages $871+ thanks to the DRAM crisis. At ~$490 the 4070 is the fastest modern 12GB card you can buy without irrational money. The catch: a $286 used RTX 3060 12GB runs every single model the 4070 runs, just 30–40% slower.

Used RTX 4070 (~$481–$519)Used RTX 3060 12GB (~$286)Used RTX 3090 24GB (~$1,286)
Best forFastest 12GB tier: 14B at ~33 tok/sSame 12GB ceiling on a budgetActual class upgrade: 27B–35B models
Bandwidth / VRAM504 GB/s / 12GB360 GB/s / 12GB936 GB/s / 24GB
The catchSame model ceiling as a card half its price14B at 25–32 tok/s, slower prompt processing2.6× the price, 350W, EPS-sized PSU needs

Honest take: Buy the used 4070 at $450–$500 if you want the 14B class to feel snappy and you also game at 1440p. If the card only runs LLMs, the $286 RTX 3060 does the same job at a readable speed and banks $220 toward a future 24GB card — the upgrade that actually changes what you can run.

Why this card is suddenly interesting again

The RTX 4070 launched April 13, 2023 at $599 and was discontinued when the 50-series arrived. Normally a two-generation-old midrange card would be a boring $300 afterthought by now. The DRAM crisis rewrote that script.

TechPowerUp’s retailer tracking shows RTX 50-series median prices jumped as much as 41% in August 2026 alone. The RTX 5070 12GB — MSRP $549 — averaged around $900 in August, up from $660 a month earlier, and sat at $871.88 on the August 31 tracker. Meanwhile the used 4070 has drifted up only gently: ResalePrices puts the market average at $509 with a fair asking range of $481–$519, and eBay sold listings through late August average $488–$525 depending on the window (30-day: $488; 90-day: $467).

That spread is the whole story. New 12GB Blackwell costs $870+. Used 12GB Ada costs ~$490. The 5070’s 672 GB/s GDDR7 is about 33% faster than the 4070’s 504 GB/s — real, but it unlocks zero additional models, because both cards hit the same 12GB wall. Paying $380 extra for speed on the same model ceiling is the kind of math that only works if someone else is buying.

If you’re new to VRAM math, run your target model + context through our VRAM calculator before buying anything — and see the GPU buying guide for how the DRAM squeeze reshaped every tier (background on the shortage itself: DDR5 and SSD prices doubled).

What a used RTX 4070 actually delivers

Specs that matter for inference: 12GB GDDR6X on a 192-bit bus at 504 GB/s, 5,888 CUDA cores, 200W TDP with a single 8-pin (or 16-pin adapter, board-dependent). Token generation on local LLMs is memory-bandwidth-bound, so 504 GB/s is the number to anchor on — 1.4× the RTX 3060’s 360 GB/s, 1.75× the RTX 4060 Ti’s 288 GB/s, and a little over half a used RTX 3090’s 936 GB/s.

Measured numbers, not vibes — Hardware Corner’s RTX 4070 benchmark page (tested at 16K context, llama.cpp):

ModelQuantVRAM fitSpeed on RTX 4070
Llama 3.1 8B classQ4_K_XL~5–6GB, comfortable~52 tok/s at 16K ctx
Qwen3 14BQ4_K_M (~9GB)Fits, tight at long ctx32.7 tok/s at 16K ctx
Qwen3.5-9BQ4_K_M (5.68GB)Comfortable + 32K ctx~50+ tok/s (8B-class speed)
Gemma 4 12B QAT~7GBComfortable8B-class speed, multimodal
Qwen2.5-Coder-14BQ4_K_M (8.7GB)Fits~30+ tok/s, FIM autocomplete

ModelFit’s page for the card agrees on the headline: 7B–9B models around 52 tok/s. For calibration, our 12GB tier guide measured the same 14B class at 25–32 tok/s on the RTX 3060 — the 4070’s ~33–36 tok/s on 14B is almost exactly what the 504/360 bandwidth ratio predicts. No magic, just a wider bus.

Reading speed is roughly 7–10 tok/s. Everything in that table is comfortably past it, which means the practical difference between the 4070 and 3060 isn’t “usable vs unusable” — it’s “instant vs brisk.” Where the 4070 pulls further ahead is prompt processing (compute-bound, and Ada’s 5,888 cores at higher clocks beat Ampere’s 3,584), so long-context RAG and coding-agent workflows with big system prompts feel noticeably less laggy.

What you cannot run — the honest section

The 12GB ceiling is binary, and no amount of bandwidth fixes it:

  • GPT-OSS 20B — 12.8GB of MXFP4 weights before KV cache. Spills to system RAM, speed craters.
  • Gemma 4 26B-A4B QAT — ~15GB. The best small MoE of 2026 needs a 16GB card.
  • Qwen3.6-27B — 16.8GB at Q4_K_M. The 24GB-tier daily driver, out of reach.
  • Qwen3.6-35B-A3B — ~21GB. The 107-tok/s speed demon on a 3090 is a slideshow here.
  • Dense 70B — 43GB at Q4_K_M. Not even partially sane; see running a 70B on 24GB for why even that is a compromise.

If any model on that list is why you’re upgrading, skip the entire 12GB tier and buy the used RTX 3090 — $1,286 average in early September 2026 ($1,222–$1,328 fair range, 343 listings). It’s the only used card under $2,000 that changes your model class rather than your token speed. Full analysis in the used RTX 3090 guide.

The trap that makes a 4070 feel broken: 14B + long context

The most common complaint from 12GB owners running a 14B: it benchmarks at 30+ tok/s on day one, then some session later it’s generating at 5 tok/s and nothing obvious changed. What happened: a 14B Q4_K_M is ~9GB of weights, and at 16K context the KV cache pushes the total past ~11.5GB. Add a browser eating VRAM and Ollama silently splits the model across GPU and CPU.

Check before you blame the card:

$ ollama ps
NAME            ID          SIZE     PROCESSOR         UNTIL
qwen3:14b       a3f...      11 GB    91%/9% CPU/GPU    4 minutes from now

Anything other than 100% GPU in the PROCESSOR column means you’re bandwidth-limited by system RAM, not the card. The fix, in order:

  1. Cap context to what you actually use: ollama run qwen3:14b then /set parameter num_ctx 8192 (or set it in a Modelfile).
  2. Enable flash attention + quantized KV cache on the service (systemd drop-in or Windows service env, not your shell): OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0 — roughly halves KV cache VRAM, which at 16K context buys back over a gigabyte on a 14B.
  3. Close Chrome. On Windows, hardware acceleration routinely holds 1–2GB of VRAM.

Re-run ollama ps and confirm 100% GPU. Speed comes back to the mid-30s. More depth on the spillover diagnosis in Ollama not using your GPU.

Used 4070 vs the alternatives, September 2026 prices

CardStreet priceBandwidthVRAM14B Q4 speedVerdict
Used RTX 3060 12GB$286 ($274–$296)360 GB/s12GB25–32 tok/sValue pick, same ceiling
Used RTX 4070~$509 ($481–$519)504 GB/s12GB~33–36 tok/sSpeed pick in the tier
Used RTX 4070 Super~$570504 GB/s12GB~same as 4070Skip: same bandwidth, +$60
Used RTX 4060 Ti 16GB$377–$470288 GB/s16GB22.4 tok/sCapacity over speed
New RTX 5070~$871–$900672 GB/s12GB~45 tok/s (est.)DRAM-crisis casualty
Used RTX 3090$1,286936 GB/s24GB40+ (also runs 27B–35B)The real upgrade

Three comparisons deserve a sentence each:

vs the 4070 Super (~$570 used, Aug 31 tracker): the Super has 7,168 CUDA cores but the identical 504 GB/s memory system, so token generation is effectively the same. Its extra compute helps gaming and prompt processing a little. For an LLM box, the $60 premium buys almost nothing — buy the cheaper non-Super.

vs the 4060 Ti 16GB ($377–$470): this is the genuinely hard call, and it’s the same trade we covered yesterday from the other side. The 4060 Ti’s 16GB runs Gemma 4 26B-A4B and GPT-OSS 20B — a real capability the 4070 lacks — but its 288 GB/s bus runs everything at RTX-3060 speed. Pick by workload: bigger models slowly (4060 Ti 16GB) or the 14B ceiling quickly (4070). If you can’t decide, that ambivalence is an argument for saving toward a 3090, which deletes the trade-off.

vs a new 5070: 33% more bandwidth, identical model ceiling, ~$380 more. At August’s trajectory (guru3d logged +39% at US retailers; TechPowerUp’s median tracker is worse) the gap is widening, not closing. There is no version of local-AI math where the 5070 at $900 beats a $490 used 4070 — you’d be most of the way to a used 3090.

Power and running cost

At 200W board power under sustained inference load, the 4070 costs about $0.038/hour at the current 18.83¢/kWh US residential average — call it $0.90 for a 24-hour batch day, and pennies for interactive chat where the card mostly idles. It runs happily on a quality 550–650W PSU with one 8-pin (some AIB boards use the 16-pin adapter). Compare a used 3090’s 350W and its fussier transient spikes; the 4070 is the card you can drop into whatever office PC you already own. Deeper math in what a 24/7 AI server actually costs, and if you do run it around the clock, power-limiting to ~150W costs only a few percent of tok/s.

Buy or skip: the price thresholds

Buy at $450–$500 if:

  • You want the 14B class (Qwen3 14B, Phi-4, Qwen2.5-Coder-14B) at 30+ tok/s with room for real context
  • The machine also games — 1440p performance is a legitimate half of the value at this price
  • You run long prompts or a local coding backend where Ada’s prompt-processing lead matters (pair it with Continue.dev or Cline against an Ollama endpoint)

Skip if:

  • Your budget is the constraint → used RTX 3060 12GB at $286. Same models, still past reading speed. Full case in the used 3060 guide.
  • You see listings at $520+ → at that point you’re paying 4070-Super money for a non-Super, and you’re $130 from a used 4060 Ti 16GB plus $700 from a 3090. Walk.
  • You want models that make 2026 exciting (26B-A4B MoEs, 27B dense, 35B-A3B) → nothing at 12GB delivers that. Used 3090, or rent a 24–48GB pod on RunPod for $0.30–$0.70/hr to find out which model you’d actually live with before spending $1,300.
  • You only need occasional big-model access → RunPod again; the rent vs buy math says buy only when usage is daily.

The self-hosting stack (Ollama v0.32.x, llama.cpp, Open WebUI — see aifoss.dev for the FOSS side) treats the 4070 as a first-class citizen: standard CUDA, no ROCm overrides, no Pascal-style driver deadline. Ada Lovelace (sm_89) sits well inside CUDA’s support window, so unlike the $260 Tesla P40 this card has no expiration date you need to plan around.

FAQ

Is the RTX 4070 good for local AI in 2026? Yes, within its tier: it’s the fastest 12GB card that costs sane money — ~52 tok/s on 8B models and ~33 tok/s on 14B at Q4_K_M (Hardware Corner, 16K context). The limit is capacity, not speed: nothing above ~14B dense fits.

Used RTX 4070 vs used RTX 3060 12GB for LLMs — which should I buy? They run the identical model list. The 4070 is 30–40% faster and much quicker at prompt processing; the 3060 costs $286 vs ~$509. If the GPU is only for LLMs and the budget matters, take the 3060 and save toward 24GB. If it also games or you live in long-context workflows, the 4070 earns its premium.

Does the RTX 4070 run Qwen3.6-27B or GPT-OSS 20B? No. Qwen3.6-27B needs 16.8GB at Q4_K_M and GPT-OSS 20B’s MXFP4 weights are 12.8GB before KV cache — both spill past 12GB. The cheapest cards that run them comfortably are 16GB (GPT-OSS 20B, Gemma 4 26B-A4B) and 24GB (Qwen3.6-27B).

Is the RTX 4070 Super worth $60 more for inference? No. Same 12GB, same 504 GB/s bus, so token generation is effectively identical. The extra cores help gaming and prefill slightly. For an inference box, buy whichever is cheaper — that’s usually the non-Super.

Why not just buy the new RTX 5070? Because the DRAM crisis pushed it to ~$871–$900 against a $549 MSRP (August 2026 trackers). It’s 33% faster on paper with the same 12GB ceiling. The $380+ difference is most of the way to a used RTX 3090, which is a real class upgrade instead of a speed bump.

Sources

Last updated September 2, 2026. Prices and specs change; verify current rates before purchasing.

Was this article helpful?