TL;DR: One pick per budget: used RTX 4060 Ti 16GB at $500, used RTX 3090 at $1,000 — and at $1,500 the answer is still the 3090, because nothing between $1,050 and $2,268 adds a single gigabyte of VRAM. At $2,500, a used RTX 4090. The new RTX 5090 is out of every tier at a $4,699 median street price.

	$500	$1,000	$1,500	$2,500
The pick	Used RTX 4060 Ti 16GB	Used RTX 3090 24GB	Used RTX 3090 24GB (yes, again)	Used RTX 4090 24GB
Street price (Sep 2026)	$377–$470	~$1,000–$1,050	$1,050 + $450 for RAM/PSU	~$2,268
Model class it unlocks	14B dense / 20B–26B MoE	27B–35B	Same — bank the difference	Same class, ~2× the speed
Measured speed	22–27 tok/s (Qwen3-14B)	107 tok/s (Qwen3.6-35B-A3B)	—	225 tok/s (GPT-OSS 20B)
The catch	288 GB/s bus is slow	2020 silicon, no warranty	Feels wrong to underspend	$669 over the 3090 tier

Honest take: If you can stretch to ~$1,050, buy a used RTX 3090 and stop reading. It has been the answer since 2024 and the 2026 GDDR7 shortage made it more true, not less.

Every GPU budget in 2026 lands on the same question: how much VRAM can you get without paying the AI-demand tax that has new cards selling 35–135% over launch MSRP? This guide gives one verdict per budget — not a 15-card listicle. Before you commit to any tier, run your target model and context length through our VRAM calculator to confirm the fit, because the wrong tier is the most expensive mistake on this page.

Prices below are tracked US street prices, checked September 4, 2026. Sources for every number are at the end.

The two specs that decide everything

A local LLM buyer’s guide needs exactly two axes, and neither is TOPS or CUDA core count:

1. VRAM capacity decides which models you can run. A model either fits or it doesn’t — a 16.8GB Q4 quant of Qwen3.6-27B on a 16GB card isn’t “slow,” it’s broken or spilling into system RAM at a tenth of the speed.
2. Memory bandwidth decides how fast the models that fit will generate. Token generation is memory-bound: the GPU reads every active weight once per token, so tokens/sec tracks GB/s almost linearly within an architecture.

That’s why this guide is a used-card guide. The 2026 GDDR7 shortage pushed new-card prices up across the board while used GDDR6/6X cards — same VRAM, same bandwidth — only drifted. Our May GPU buying guide covered the full theory; this page is the September verdict sheet.

$500: Used RTX 4060 Ti 16GB — the cheapest modern 16GB

The pick: a used RTX 4060 Ti 16GB at $377–$470 on the used market (GPUPrix tracking, August–September 2026).

Sixteen gigabytes is the floor where local AI stops feeling like a demo. It fits Gemma 4 26B-A4B QAT (~15GB, Google’s official quantization-aware checkpoints), GPT-OSS 20B (~45 tok/s on this card per WillItRunAI’s measurements), and Devstral Small 2 at Q4_K_M (14.7GB) — a genuinely useful coding model if you pair it with Continue.dev and Ollama.

The catch is the 288 GB/s memory bus. Hardware Corner measured Qwen3-14B Q4_K_M at 22.4 tok/s; Smeltcore got 27.4 tok/s at 4K context. That’s RTX 3060 speed with four extra gigabytes — you’re buying capacity, not velocity. Above reading speed, below “instant.”

Why not alternatives at this price:

• A new RTX 5060 Ti 16GB is the same VRAM at $626–$805 (videocardprices, August 2026) — up roughly 39% since June. It’s 55% faster (448 GB/s), but at street price you’re most of the way to a used 3090, which embarrasses both.
• A used RTX 3060 12GB at ~$286 (ResalePrices) is the true floor: 42 tok/s on 8B models, 23 tok/s on 14B. If $500 is genuinely the ceiling and you want change left over, it’s the most VRAM per dollar on the market — but 12GB locks you out of the 20B–26B class that makes 2026’s models interesting.
• Anything 8GB — used 3070, new RTX 5060 — is a dead end. The models worth running start above 8GB, full stop.

Full teardown of this pick: Used RTX 4060 Ti 16GB — worth it? and the companion best local LLMs for 16GB VRAM.

$1,000: Used RTX 3090 — still the answer, six years in

The pick: a used RTX 3090 at roughly $1,000–$1,050 on eBay (Best Value GPU and GPUDojo tracking, September 2026).

The 3090 is the entire reason the used market matters for local AI: 24GB of GDDR6X at 936 GB/s — 2020 flagship silicon at a 2026 mid-range price. Twenty-four gigabytes is where the current best local models actually live:

• Qwen3.6-27B (16.8GB at Q4) — 77.2% on SWE-bench Verified, ~40 tok/s on a 3090, ~60 with multi-token prediction enabled (InsiderLLM’s measured run)
• Qwen3.6-35B-A3B (21GB at Q4) — the MoE speed demon: 107 tok/s through Ollama on this exact card
• Everything in the 16GB tier, now with 32K+ context headroom instead of gasping at 8K

Here’s what that looks like in practice — this is the whole pitch in one command:

$ ollama run qwen3.6:35b-a3b --verbose "Explain PCIe lane bifurcation in two paragraphs"
...
eval rate:            107.36 tokens/s

The trade-offs, honestly: it’s a used 2020 card with no warranty, it pulls 350W under load (about $0.042/hour at the $0.12/kWh US average — the power math is real but small), and it wants a quality 850W PSU. Buy from a seller with real feedback, run nvidia-smi and a 30-minute inference soak on day one, and you’ve de-risked most of it.

We’ve made the full case for the used 3090 before, and the 24GB model guide shows exactly what you’ll run. Nothing at this price is close: the 16GB cards can’t load the 27B class, and AMD’s used RX 7900 XTX (~$815) saves $200 but pays a CUDA-compatibility tax in tooling that most people should not sign up for.

$1,500: Still the used RTX 3090 — and that’s the honest verdict

This is the tier where a buyer’s guide is supposed to hand you something shinier. There isn’t anything.

Between the 3090 at $1,050 and the used 4090 at $2,268, no card adds VRAM. What the market offers in the gap:

Card (Sep 2026 street)	VRAM	Bandwidth	Why it loses to the 3090
Used RTX 3090 Ti, ~$1,240	24GB	1,008 GB/s	+8% bandwidth for +$190 and 450W
RTX 5070 Ti new, $919–$1,233	16GB	896 GB/s	Loses 8GB — drops the whole 27B–35B class
RTX 5080 new, $1,400+	16GB	960 GB/s	Same 16GB ceiling at an even worse price

A used RTX 3090 Ti is the only defensible splurge — GPUDojo tracks it around $1,240 — but paying $190 for 8% more tokens/sec and 100W more heat is not a decision, it’s an impulse. The 16GB Blackwell cards are faster per gigabyte they have, but model fit beats speed every time: a $800 used 4070 Ti Super generating fast on models a 3090 also runs, while being locked out of the models it can’t — that’s the definition of parked money.

So the $1,500 verdict: buy the $1,050 RTX 3090, and put the remaining ~$450 into the platform — 64GB of DDR5 (for MoE models that overflow, and for CPU-offload headroom) and a quality 850W PSU beat any GPU upgrade available in this window. Or bank it: you’re $770 of patience away from the next real tier.

The problem that actually defines this tier: readers regularly buy a 16GB card here, then hit CUDA error: out of memory the first time they load Qwen3.6-27B Q4 at 8K context — 16.8GB of weights plus KV cache doesn’t fit in 16GB. You can quantize the KV cache and shrink context to limp through (every OOM fix, ranked), but the real fix is buying the right tier once. Check your exact model + context combination in the VRAM calculator before you spend.

$2,500: Used RTX 4090 — the speed tier

The pick: a used RTX 4090 at ~$2,268 on eBay (Best Value GPU, August 2026 average; ResalePrices tracks $2,150–$2,350).

Same 24GB as the 3090, so it runs the same models — what you’re buying is velocity: 1,008 GB/s of bandwidth plus two generations of compute. The llama.cpp community benchmark thread has it at ~225 tok/s on GPT-OSS 20B (MXFP4), and it roughly doubles the 3090 on prompt processing, which is what you feel in long-context and agentic work. If your workload is a coding agent hammering 32K contexts all day, the 4090 is the difference between waiting and working — for chat, it’s a luxury.

The uncomfortable math: a used 4090 costs more today than its $1,599 launch MSRP, four years after release. That’s the 2026 GPU market in one sentence. Whether that premium is worth it over the 3090 comes down to how many hours per day the card is actually generating.

What about the RTX 5090? Its 32GB would genuinely unlock the 70B-at-IQ3 class — but the median US street price hit $4,699.99 in September 2026 (videocardprices), up from $4,299.99 in June and 135% over the $1,999 launch price. It’s not in this guide because it’s not in any sane budget. If you need more than 24GB occasionally, don’t buy it — rent it: an 80GB A100 or H100 by the hour on RunPod covers the rare 70B+ job for single-digit dollars, and our rent-vs-buy math shows exactly where the crossover sits. Dual used 3090s at ~$2,100 are the other 48GB path if you’d rather own — see the 48GB tier guide.

Cards that didn’t make any tier

• Used Tesla P40 ($260, 24GB): tempting VRAM/dollar, but Pascal loses driver support mid-2028 — a depreciating clock.
• Used RTX A5000 ($2,435): the 3090’s silicon at twice the price for a blower format — rack builders only.
• Mac / unified memory: a different (good) answer for a different budget — the 128GB unified memory guide covers it. This page is discrete-GPU truth only.

FAQ

Is a used GPU safe for AI workloads? Safer than the horror stories suggest. Inference is a lighter duty cycle than 2021-era mining. Test on arrival: nvidia-smi for recognized VRAM, then a 30-minute sustained inference run watching temps. Most failures announce themselves in the first hour.

Why is every pick NVIDIA? CUDA is still where llama.cpp, vLLM, ExLlama, and ComfyUI development lands first. AMD’s ROCm has improved (our Ubuntu setup guide works), but on the used market the 3090 is only ~$200 over the RX 7900 XTX — too small a discount for the friction.

Should I wait for prices to drop? The supply-side signals point the wrong way: DRAM contract prices are up sharply and reports say NVIDIA ships no new consumer GPUs in 2026. Used Ampere/Ada prices have been flat-to-up all year. Waiting has been a losing trade for 18 months.

What about two cheaper cards instead of one good one? Splitting a dense model across two cards over PCIe adds latency and complexity for little decode gain — multi-GPU math here. One big card first, always. The exception is dual 3090s as the budget 48GB play once you’ve outgrown 24GB.

I only have $300. Used RTX 3060 12GB at ~$286. It runs 8B models at 42 tok/s and that’s a real local AI experience. Skip 8GB cards entirely.

Recommended Gear

The four verdicts, in one place:

• $500 tier: Used RTX 4060 Ti 16GB — $377–$470 used
• $1,000 and $1,500 tiers: Used RTX 3090 24GB — ~$1,050 used
• $2,500 tier: Used RTX 4090 24GB — ~$2,268 used
• Budget floor: Used RTX 3060 12GB — ~$286 used
• Don’t want to buy at all? Rent by the hour on RunPod

Sources

• RTX 3090 Price Tracker US, Sep 2026 — Best Value GPU
• RTX 3090 24GB Used Price & History, Sep 2026 — GPUDojo
• RTX 4090 Price Tracker US — Best Value GPU
• RTX 4090 Used Price & Fair Asking Range — ResalePrices
• RTX 5090 Price Tracker, September 2026 — videocardprices.com
• RTX 5090 Price in 2026: Current Prices and Price History — TrackaLacker
• RTX 4060 Ti 16GB used price comparison — GPUPrix
• RTX 4060 Ti 16GB LLM benchmarks — Hardware Corner
• Qwen3-14B on RTX 4060 Ti 16GB — Smeltcore
• Qwen3.6-27B with MTP on RTX 3090 — InsiderLLM
• llama-bench: Qwen3.6-27B on RTX 3090 — ahelpme
• GPT-OSS benchmark thread — llama.cpp discussions #15396
• GPT-OSS 20B VRAM requirements — WillItRunAI
• Gemma 4 QAT checkpoints — Unsloth docs
• RTX 5060 Ti price tracker — videocardprices.com
• RTX 3090 Ti Used Price & History — GPUDojo
• Asus Prime RTX 5070 Ti at $900 — Tom’s Hardware
• Used RTX 3060 pricing — ResalePrices

Last updated September 4, 2026. Prices and specs change; verify current rates before purchasing.