How to Self-Host Flux 2 in 2026: Real VRAM Numbers for Every GGUF Tier, and Whether to Upgrade From Flux.1-dev

By RunAIHome Team · 12 min read
fluxcomfyuiimage-generationgpuvramgguftutorial

TL;DR: Flux 2 is really three models: the 32B FLUX.2 [dev] (needs 19-20GB just for a Q4 GGUF), and the Klein 4B/9B distills that actually fit consumer cards. On 24GB, run dev at Q4_K_M; on 16GB or less, run Klein and stop feeling bad about it — the 4B is the only Apache 2.0 weight in the family anyway.

What you’ll be able to do by the end:

  • Pick the right Flux 2 checkpoint and quant for your exact VRAM (8, 12, 16, 24, or 32GB), from verified file sizes — not vibes
  • Set up the ComfyUI workflow: transformer, Mistral text encoder, VAE, correct folders, and the update step everyone skips
  • Decide with actual benchmark data whether Flux 2 output justifies replacing a working Flux.1-dev pipeline

Honest take: If you have 24GB and image quality pays your bills, the upgrade is real — Flux 2 dev wins 66.6% of head-to-head text-to-image comparisons and finally renders legible text. If you’re on 16GB or less and Flux.1-dev works for you, Klein 4B is the only variant worth the download, and even that’s optional.

Black Forest Labs shipped FLUX.2 [dev] on November 25, 2025 — a 32B-parameter text-to-image and editing model, more than 2.5× the size of Flux.1-dev’s 12B. The [klein] family followed on January 15, 2026: a 4B under Apache 2.0 and a 9B under the FLUX non-commercial license, both distilled from dev. Eight months on, the quantization ecosystem has settled — Unsloth and city96 GGUFs cover every tier from 12.9GB to 35GB, ComfyUI runs everything natively, and there’s enough community benchmark data to answer the question that actually matters: what fits on your card, and is it better than what you’re already running?

This is the guide we wished existed when Flux.1 first landed: file sizes from the actual repos, generation times from real hardware, and the license fine print up front.

The Flux 2 family, sorted by what you can legally do with it

ModelParamsLicenseReleasedRealistic minimum VRAM
FLUX.2 [dev]32BFLUX.2-dev Non-CommercialNov 25, 202516GB (Q3 GGUF, tight)
FLUX.2 [klein] 9B9BFLUX Non-CommercialJan 15, 202616GB (FP8)
FLUX.2 [klein] 4B4BApache 2.0Jan 15, 20268GB

The license column deserves more attention than it usually gets. Only the Klein 4B is Apache 2.0 — output you can sell, weights you can fine-tune commercially, no strings. Both the 32B dev and the 9B Klein are non-commercial: fine for a home lab, not fine for client work without a paid BFL license. We covered why this trips people up in our local image-gen licensing breakdown — the short version is that the best-looking checkpoint in a family is almost never the commercially usable one.

One architectural change matters for your hardware planning: Flux 2 replaces Flux.1’s T5-XXL/CLIP text encoders with Mistral 3 Small. That’s a full LLM riding shotgun, and it’s why the system RAM floor moved: 32GB minimum, 64GB recommended, with roughly 60GB of disk for a full dev setup (transformer + encoder + VAE).

FLUX.2 dev: the real VRAM table

The 32B dev transformer weighs about 64GB in BF16 — H100 territory, not consumer hardware. The Comfy-Org FP8 build (made with NVIDIA) is about 35GB as a file, which still doesn’t fit any consumer card whole. What makes dev runnable at home is GGUF. Verified file sizes from the Unsloth FLUX.2-dev-GGUF repo, with city96’s Q4_K_S alongside:

QuantFile sizeFits fully on
Q8_035 GBNothing consumer (offload required)
Q6_K27.4 GBRTX 5090 32GB
Q5_K_M23.9 GBRTX 5090 (4090 with offload)
Q4_K_M20 GBRTX 4090 / 3090 24GB
Q4_K_S19 GBRTX 4090 / 3090 24GB
Q3_K_M15.8 GB16GB cards, tight
Q2_K12.9 GB16GB cards comfortably; quality suffers

Mapping that to actual cards:

24GB — RTX 4090 and RTX 3090. Q4_K_M (20GB) or Q4_K_S (19GB) is the sweet spot: transformer fully resident, text encoder offloaded to system RAM. Community benchmarks on an RTX 4090 running the Q8_0 GGUF with partial offload clock 15-17 seconds per 1024×1024 image at 20 Euler steps — Q4 runs faster since more of the model stays on-die. That’s your reference number for “is my setup healthy.”

32GB — RTX 5090. Q5_K_M (23.9GB) or Q6_K (27.4GB) resident with room for the VAE. Blackwell cards also get the NVFP4 path — the FLUX.2-dev-NVFP4 build is a ~14GB footprint with hardware-accelerated 4-bit math, and it’s the fastest way to run dev on a 5090. We benchmarked that pipeline separately in the NVFP4 ComfyUI speed guide.

16GB — RTX 5060 Ti 16GB, 5080, 4080. Q3_K_M at 15.8GB technically loads, but you’re one context spike from an OOM. Q2_K (12.9GB) fits with headroom and visibly degrades fine detail — at which point Klein 9B at FP8 gives you better output from a model that was actually designed for this tier. That’s the honest recommendation. Being repeatedly one gigabyte short is a hardware problem rather than a quant problem — the 2026 GPU buyer’s guide covers the 24GB step up.

12GB and 8GB. Dev is off the table. Klein territory — next section.

Klein: what 8-16GB cards should actually run

The Klein models aren’t crippled afterthoughts; they’re distills BFL built specifically for consumer hardware, and the speed numbers are the appeal:

  • Klein 4B: ~13GB file in BF16, ~7GB in FP8. Per BFL’s own repo, it fits in ~8GB VRAM, and it generates a 1024×1024 image in about 1 second on an RTX 4090. Sub-second to one-second generation changes how you use the tool — iteration becomes interactive instead of batch-and-wait.
  • Klein 9B: needs ~29GB of working VRAM at full FP16 (that’s 5090/Mac territory), but FP8 brings it to 16GB cards. On an RTX 4090 at FP16 it generates in ~3.7 seconds at 28 steps — the quality-to-latency sweet spot of the family.

On a 12GB card like the RTX 3060, Klein 4B BF16 is tight but workable with the encoder offloaded; FP8 is comfortable. On 8GB, use FP8 or the GGUF builds and expect encoder offload to add a couple of seconds per prompt on the CPU side.

If your card predates this tier entirely, Flux.1-dev still runs on 8GB via GGUF Q4 at ~6.8GB — Flux 2 didn’t retroactively make it worse.

Which card to run it on

Street prices on these cards move week to week, so the links go to current listings rather than a number this article hasn’t verified:

Your situationThe cardWhat you can runWhere
You want FLUX.2 dev at full quality without offload gymnasticsRTX 5090 32GBdev, comfortablyCheck price
You have 24GB and are willing to manage VRAMRTX 4090 24GBdev with offloadCheck price
8-16GB card - Klein is what you should actually runRTX 5060 Ti 16GBKleinCheck price
You want to test before downloading 60GBRented RTX 4090eitherRunPod - $0.34-0.69/hr

ComfyUI setup: files, folders, and the update everyone skips

ComfyUI supports Flux 2 natively — no custom nodes for the standard workflow (GGUF loading still uses the ComfyUI-GGUF extension). Three files, three folders:

ComfyUI/models/
├── diffusion_models/
│   └── flux2-dev-Q4_K_M.gguf              # or your chosen quant
├── text_encoders/
│   └── mistral_3_small_flux2_fp8.safetensors
└── vae/
    └── flux2_vae.safetensors

Grab the transformer quant from Unsloth or city96, and the text encoder + VAE from the Comfy-Org flux2-dev repo (split_files/ directory). Take the FP8 text encoder, not BF16 — it’s a 24B-class Mistral encoder, and the BF16 version alone can eat more memory than your transformer.

huggingface-cli download unsloth/FLUX.2-dev-GGUF \
  flux2-dev-Q4_K_M.gguf \
  --local-dir ComfyUI/models/diffusion_models/

Then update ComfyUI before you load anything. Flux 2 node support shipped in late-November 2025 builds, and the GGUF loader gained FLUX.2 architecture support in ComfyUI-GGUF updates after that (the feature request thread is where to check current status). A stale install fails with model-detection errors that look like corrupt downloads but aren’t.

On a healthy launch you’ll see the model land where it should:

$ python main.py
Total VRAM 24564 MB, total RAM 65536 MB
Using pytorch attention
loaded completely ... flux2-dev-Q4_K_M.gguf
Requested to load MistralTEModel
loading in lowvram mode

That lowvram mode line on the text encoder is correct behavior, not a warning — the Mistral encoder runs once per prompt, so parking it in system RAM costs you a second or two per generation and saves ~12GB of VRAM.

The OOM you’ll actually hit, and the fix

The most common failure on 24GB cards looks like this mid-generation:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.50 GiB.

Three fixes, in order of how often they work: (1) you downloaded the BF16 text encoder — swap it for mistral_3_small_flux2_fp8.safetensors; (2) launch with --lowvram so ComfyUI streams weights instead of front-loading them; (3) drop one GGUF tier (Q4_K_M → Q4_K_S buys you a gigabyte). And if you’re on Windows and the process dies with OS error 1455 instead of a CUDA error, that’s your paging file, not your GPU — we have a dedicated fix for that one.

Is it actually better than Flux.1-dev?

The upgrade question deserves numbers, not marketing. What the data says:

  • In fal.ai’s head-to-head win-rate testing, FLUX.2 dev wins 66.6% of text-to-image comparisons versus 48.1% for comparable models — a consistent, measurable gain over Flux.1 and contemporary open-weight systems, not a coin flip.
  • Text rendering is the headline fix. Flux.1’s misspelled-poster problem is mostly gone: Flux 2 produces legible, consistent text on posters, packaging, and UI mockups. If your use case involves any in-image text, this alone justifies the switch.
  • On quality-vs-cost ELO charts, FLUX.2 dev clusters in the 1030-1050 band — near-frontier quality at open-weight prices.
  • Editing is built in. Flux 2 dev handles multi-reference editing natively, where Flux.1 needed the separate Kontext model for image-conditioned edits.

Now the other side of the ledger. Flux.1-dev runs in ~12GB at FP8; the equivalent-quality Flux 2 dev quant needs 19-20GB plus a heavier encoder and twice the system RAM. You’re paying a full GPU tier for the quality jump. On a 24GB card that’s fine. On 16GB, “upgrading” means either a quant tier that erases the quality gains (Q2_K) or switching to Klein 9B — which is a sidegrade with better text rendering, not a straight upgrade.

So: 24GB+ and quality-sensitive work → upgrade, keep Q4_K_M as your daily driver. 16GB or less with a working Flux.1 pipeline → try Klein 4B for the speed, keep Flux.1-dev for finals, skip dev entirely.

No GPU, or want to test before committing 60GB of downloads?

A RunPod RTX 4090 instance runs about $0.34-0.69/hour depending on tier — two dollars of testing tells you whether Flux 2 dev output justifies rearranging your hardware budget, and RunPod publishes an official FLUX.2 deployment guide with a ready ComfyUI template. If you find yourself renting more than ~10 hours a month, the rent-vs-buy math starts favoring a used 3090.

FAQ

Can I use Flux 2 dev output commercially? Not under the default license — FLUX.2 [dev] is non-commercial, same as the 9B Klein. The Klein 4B is Apache 2.0, and its output is yours to sell. BFL sells commercial licenses for the dev weights.

Does Flux 2 work on a used RTX 3090? Yes, and it’s arguably the value play: 24GB fits Q4_K_M (20GB) the same as a 4090, just slower. Ampere doesn’t get FP8 or NVFP4 acceleration, so GGUF is the right format for a 3090.

How much system RAM do I really need? 32GB minimum, 64GB recommended. The Mistral text encoder offloads to system RAM in every consumer-VRAM workflow, and Q4-class transformer + FP8 encoder + OS overhead lands right around the 32GB line.

Do I need custom nodes in ComfyUI? Native safetensors workflows: no, just a current ComfyUI build. GGUF quants: yes, the ComfyUI-GGUF custom node, updated to a version with FLUX.2 support.

Is Klein 4B better than Flux.1-dev? Different trade. Flux.1-dev (12B) still edges out Klein 4B on raw detail; Klein generates in ~1 second on a 4090, renders text far better, and is Apache 2.0 (Flux.1-dev is non-commercial). For commercial output or rapid iteration, Klein wins. For maximum-quality personal work on 12-16GB, Flux.1-dev remains competitive.

What about Apple Silicon? The GGUF quants load through ComfyUI on M-series unified memory — a 48GB+ M4 Max fits Q8_0 outright. Generation is materially slower than a 4090; Klein 9B is the more sensible Mac daily driver.

Sources

Last updated August 9, 2026. Prices, quant availability, and ComfyUI support status change quickly; verify current requirements before downloading 60GB of weights.

  • RTX 4090 — the 24GB workhorse for FLUX.2 dev Q4_K_M
  • RTX 5090 — 32GB for Q5/Q6 quants and the NVFP4 fast path

Was this article helpful?

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.