Every “best local AI model” article skips the question that actually matters: best for what VRAM. A model that runs beautifully on a 4090 is irrelevant to someone with a laptop iGPU; a model that fits on 6 GB is left on the table by people with 24 GB to spare. This guide goes by tier, listing the practical best-in-class picks for language, image, and audio at each common GPU memory size as of mid-2026.

For each tier I will name the realistic GPUs that hit it, what you can run comfortably, and what stretches the limit. All models assumed at sensible quantization (typically Q4_K_M for LLMs, fp8 for image, default for audio) — see our quantization explainer for the math.

4 GB — laptop integrated graphics, oldest dGPUs

Cards: Apple M1/M2/M3 base (8 GB unified, ~4 GB usable for AI), older entry-level cards like GTX 1650.

Realistic for:

• Language: Llama 3.2 1B (Q4_K_M, ~700 MB), Phi-3-mini (3.8B Q4, ~2.3 GB), Qwen 2.5 1.5B / 3B (Q4_K_M).
• Image: SD 1.5 with --lowvram flag, 512×512 only.
• Audio: Whisper.cpp tiny or base for transcription.

Stretch: With CPU offload (GGUF), you can technically run Llama 3.1 8B Q4 by spilling half the layers to system RAM. Speed will be 1–3 tokens/sec, which is enough for casual chat but tedious for anything else.

Honest take: This tier is for “the model is on my computer, I can read its output,” not for daily AI workflows. Lean into smaller, faster, well-distilled models — the gap between 3B and 8B at this VRAM tier is much less than the gap in inference speed.

6–8 GB — entry consumer (RTX 3050, 3060, 4060)

Cards: RTX 3050 8GB, RTX 3060 8GB, RTX 4060, GTX 1080, M1/M2 Pro 16 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q4_K_M (~5 GB) — the canonical “8 GB tier” model. Mistral 7B, Qwen 2.5 7B, Phi-3 medium, Gemma 7B.
• Image: SD 1.5 comfortably; SDXL with --medvram flag (slower, but works).
• Audio: Whisper.cpp small or medium.

Stretch: Llama 3.1 8B Q5_K_M (~6 GB) for slightly higher quality. Mixed-precision SDXL on 8 GB. Llama 3.1 70B with heavy CPU offload (very slow but functional).

This is the “first real local AI” tier. Llama 3.1 8B at Q4_K_M genuinely covers most chat, coding, summarization, and document Q&A use cases. SD 1.5 plus a good LoRA gives respectable image generation. You will not be running Flux or 70B models, but everything real-world useful is on the table.

12 GB — mid-tier (RTX 3060 12GB, RTX 4070, M-series 16–24 GB)

Cards: RTX 3060 12GB, RTX 4070, RTX 5070 12GB, M3/M4 Pro 24 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q8_0 (~9 GB) — full quality 8B with comfortable context. Llama 3.2 11B Vision Q4. Qwen 2.5 14B Q4 (~9 GB). Mistral Small 22B Q3.
• Image: SDXL natively, full 1024×1024, with room for ControlNet stacks. SDXL Turbo blazing fast.
• Audio: Whisper.cpp large-v3 (~3 GB) for high-quality transcription.

Stretch: Llama 4 Scout with very aggressive quant + CPU offload (still slow). Flux dev fp8 with image-only generation at the limit. Llama 3.1 70B Q2 (compromised quality but runs).

This is the inflection tier. 12 GB unlocks SDXL natively and lets 8B LLMs run at higher precision. The quality jump from “8 GB Q4 8B” to “12 GB Q8 8B + SDXL” is large enough that this is where most enthusiasts settle.

16 GB — the 2026 sweet spot (RTX 4060 Ti 16GB, RTX 5060 Ti 16GB)

Cards: RTX 4060 Ti 16GB, RTX 5060 Ti 16GB, RTX 4070 Ti Super, M3/M4 Pro 32 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q8 with 32K context (~12 GB). Qwen 2.5 14B Q5 (~10 GB) at reasonable context. Llama 3.2 11B Vision Q8 with images.
• Image: SDXL with multiple LoRAs + ControlNet at 1024×1024 plus an upscaler stage. Flux dev fp8 (slow but works) or Flux schnell.
• Audio: Whisper large-v3 plus a TTS model like Coqui or XTTS in parallel.
• Multimodal: Run an LLM and SD/Flux concurrently for full-stack pipelines (chatbot that draws).

Stretch: Flux dev fp16 (just barely fits, no headroom). Llama 3.3 70B Q3_K (compromised quality, ~30 GB so requires heavy CPU offload).

This is the “smartest dollar in 2026” tier. RTX 5060 Ti 16GB is the card I would recommend to anyone asking “what should I buy for local AI” who is not comfortable spending $1000+. It runs everything in this tier comfortably and most things in the next tier with quantization. For the Stable Diffusion / SDXL / Flux comparison, this is the threshold where Flux becomes practical.

24 GB — the “no compromises” consumer tier (RTX 3090, 4090, 5090)

Cards: RTX 3090, RTX 3090 Ti, RTX 4090, RTX 5090, RTX 6000 Ada (48 GB).

Realistic for:

• Language: Llama 3.1 8B Q8 with 128K context. Qwen 2.5 32B Q4 (~17 GB). Llama 3.2 90B Vision Q3 (compromised). Llama 3.3 70B Q2 (very compromised).
• Image: Flux dev fp16 comfortably, with LoRAs and ControlNet. SDXL with the entire refiner + upscale pipeline in a single workflow.
• Audio: All Whisper variants plus voice cloning models.
• Combined: A 13B LLM + SDXL running concurrently for an interactive multi-modal app.

Stretch: Llama 3.1 70B Q2 (technically fits, quality is poor — go to two GPUs or 48 GB single GPU instead). Llama 4 Scout with aggressive offload.

This tier eliminates compromises for everything except 70B+ language models at full quality. It is overkill for anyone whose primary use is 8B LLM chat, sufficient for serious image work including Flux, and the entry point for “running a 13B–32B model with generous context.” Used 3090s remain the best price/VRAM ratio in this bracket.

48 GB — prosumer (RTX A6000, RTX 6000 Ada, dual 3090/4090)

Cards: RTX A6000 (48 GB), RTX 6000 Ada (48 GB), 2× RTX 3090 (~48 GB combined), Mac Studio M2 Ultra 64 GB.

Realistic for:

• Language: Llama 3.1 70B Q4_K_M (~40 GB) at full quality with reasonable context. Qwen 2.5 72B Q4. Mistral Large 123B Q3.
• Image: Multiple Flux instances or batch generation at scale.
• Combined: 70B LLM + Flux concurrently — the “do everything” tier.

Stretch: Llama 3.1 70B Q8 (~75 GB — too much for single 48 GB, requires multi-GPU). Llama 4 Scout (~62 GB Q4 — fits with very tight budget).

The 70B comfort threshold. Below 48 GB, every 70B inference is a compromise on quantization or context. At 48 GB, 70B Q4_K_M runs cleanly with 16K+ context, which puts the model at near-FP16 quality. For research or serious local development, this is the target.

80 GB — datacenter consumer (A100, H100, H200)

Cards: A100 80GB, H100, H200 (141 GB).

Realistic for:

• Language: Llama 3.1 70B Q8 (~75 GB) at full quality. Qwen 2.5 72B Q8. Llama 4 Scout Q4 with comfortable headroom.
• Image: Multi-Flux pipelines at production scale. Training small LoRAs on Flux.
• Multimodal: Llama 3.2 90B Vision Q4 with full context.

Stretch: Llama 4 Scout fp16 (~218 GB — multi-GPU only).

Mostly relevant if you have access through cloud providers (RunPod, Vast.ai, Lambda Cloud). Direct ownership of an H100 is rare outside professional deployments, but renting one for a day of intensive work is increasingly common — $1.50–$3/hour gets you everything on this tier.

What you actually need

The honest summary, by use case:

Use case	Minimum useful tier	Recommended tier
Chatbot for personal coding / writing	8 GB	12–16 GB
Casual image generation (SD 1.5 / SDXL)	8 GB	12–16 GB
Serious image generation (Flux, big LoRAs)	16 GB	24 GB
Local RAG over your documents	8 GB	12 GB
Voice transcription (Whisper)	4 GB	8 GB
Local agent (LLM + tool-use + memory)	12 GB	16 GB
70B-class LLM at full quality	48 GB	80 GB or cloud
Concurrent LLM + image + audio	16 GB	24 GB

For 2026 buyers, the dominant answer is “16 GB — 5060 Ti 16GB or 4060 Ti 16GB.” It hits the sweet spot for SDXL, runs Flux with fp8, and is more than enough for 8B-class LLMs with generous context. Going below 16 GB is a real compromise; going above 16 GB only makes sense if you specifically want 13B+ language models or comfortable Flux fp16.

Companion reads

For the actual model installation steps:

• Setting up ComfyUI on Windows — image generation side
• Ollama vs LM Studio vs llama.cpp comparison — pick a runner for language models
• How much VRAM you need for Llama models specifically — the per-model deep dive

The shortest path from “I have a GPU” to “I’m running a model”: pick your tier here, pick a runner from the comparison guide, pull a model, and go.