TL;DR: Unsloth Desktop (v0.1.800-beta, August 14, 2026) is the first mainstream local AI app that trains models, not just runs them — LLMs, diffusion, and TTS in one GUI. Inference is llama.cpp under the hood, so running models is no faster than what you already have. The training floors are the news: QLoRA on an 8B needs ~6GB, a 14B needs ~8.5GB, and a 32B needs 26GB — more than a 24GB card has.

	Unsloth Desktop	Ollama + ComfyUI	RunPod cloud fine-tune
Best for	Training on the GPU you own, one app for run + tune	Serving models daily, mature image workflows	32B/70B QLoRA your card can’t hold
Price / Cost	Free, open source	Free, open source	A100 80GB by the hour
The catch	Beta; inference = llama.cpp speeds, nothing faster	Neither one trains anything	You rent forever, own nothing

Honest take: keep Ollama for serving and ComfyUI for image work — Unsloth Desktop doesn’t beat either at their own game. Install it for the one thing nothing else on your machine does: fine-tuning a 7B–14B model on a consumer card without touching Python. On a 12GB GPU that’s a real capability you didn’t have in July.

Unsloth spent three years as the Python library everyone used for cheap fine-tuning and the GGUF quants everyone downloaded from Hugging Face. In August 2026 it became something else: the repo’s own description now reads “Local UI to run and train LLMs and diffusion models,” and the first desktop builds shipped August 10–11 with the tagline “first desktop app to run and train AI models locally.” That claim survives scrutiny — every mainstream local AI app of the last two years (Ollama, LM Studio, Jan, AnythingLLM) is an inference app. This one trains.

Whether it deserves space on your machine comes down to VRAM math, and the marketing doesn’t lead with the number that matters most: the model size your card can actually fine-tune is roughly one tier below the model size it can run. Here’s the whole picture, verified against Unsloth’s own documentation, the GitHub release notes, and the failure reports.

What shipped, exactly

The release cadence has been aggressive even by 2026 standards. From the official release log:

• v0.1.70-beta / v0.1.61-beta (Aug 10–11) — first public desktop builds for Windows, macOS, and Linux. No-code training for LLMs, diffusion, TTS, and embedding models (for zero-shot TTS that needs no training at all, see MOSS-TTS voice cloning in ComfyUI); OpenAI-compatible API for local models; Muse Glimmer 30B support; preliminary image diffusion; MiniMax-H3 video generation.
• v0.1.701-beta (Aug 11) — fixed Windows download throttling (a 200× improvement — early Windows adopters were watching 20GB models crawl), fixed AMD Strix Halo GPU detection, fixed a macOS command-line-tools dialog loop.
• v0.1.702-beta (Aug 13) — tool calling with self-healing repair of malformed function calls, web search integration, user-tunable VRAM usage, diffusion and video-generation improvements.
• v0.1.800-beta (Aug 14) — day-zero Qwen3.8-27B support (runs on 17GB of RAM/VRAM via Dynamic GGUFs), Qwen3.8-27B training with NVFP4 quantization, a claimed 10% faster GGUF inference, improved AMD RDNA 3/4 and Strix Halo support, and custom llama.cpp arguments in model settings.

Four releases in five days, each fixing real breakage from the one before. That’s the texture of a beta, and you should expect it to behave like one.

Install is one command. macOS, Linux, or WSL:

$ curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

> irm https://unsloth.ai/install.ps1 | iex

There’s also Unsloth Studio, the open-source web-UI sibling that runs headless on a server or a cloud GPU:

$ unsloth studio -H 0.0.0.0 -p 8888
# → serving at http://127.0.0.1:8888

Desktop is the packaged app; Studio is the same interface self-hosted in a browser. Both are free and open source.

The backend answer: it’s llama.cpp

The first question the queue of skeptical home-labbers asks about any new “run local models” app: new engine, or wrapper? Verified from the project README: inference is llama.cpp, with the backend selectable in Settings — CPU, CUDA, ROCm, or Vulkan — plus Metal on Apple Silicon. v0.1.800 even exposes raw llama.cpp arguments per model, which is the most honest tell of all.

That settles expectations for speed. Your GPU will decode GGUFs at the same class of tok/s it already does in anything llama.cpp-based — an RTX 3090 pulling ~41 tok/s on Qwen3.8-27B Q4_K_M in llama.cpp will do roughly the same here. For calibration on the wrapper question generally: a March 2026 head-to-head measured bare llama.cpp at 137.79 tok/s token generation where Ollama managed 122.07 tok/s on identical hardware — about 13% — because wrappers ship conservative defaults. Unsloth’s exposed llama.cpp args put it closer to the bare-metal end of that spread, and the v0.1.800 notes claim a further 10% GGUF speedup with lower VRAM use. Treat that 10% as a vendor number until independent benches land; either way, nobody should install this app for inference speed.

What the inference side does add: search-download-run for GGUF, LoRA adapters, and safetensors in one place, file upload (images, audio, PDFs), code execution, and that OpenAI-compatible API — so it can stand in for Ollama as a backend for local coding stacks. Our sister site covers wiring local models into coding tools at aicoderscope.com.

The training floors: what your card can actually fine-tune

This is the table that should decide whether you install it. Unsloth’s own requirements documentation puts approximate VRAM floors on QLoRA (4-bit) and LoRA (16-bit) fine-tuning:

Model size	QLoRA (4-bit) needs	LoRA (16-bit) needs	Cheapest card that does the QLoRA
8B	6 GB	22 GB	Any used 8GB card; 12GB is comfortable
14B	8.5 GB	33 GB	RTX 3060 12GB class
32B	26 GB	76 GB	Nothing with 24GB — needs 32GB+
70B	41 GB	164 GB	2× 24GB cards, or rent

Three verdicts fall straight out of it:

A 12GB card is a real fine-tuning machine now. The 14B QLoRA floor of 8.5GB fits an RTX 3060 12GB with headroom. Pair that with the fact that 7B–14B models are exactly the size class where a domain-specific fine-tune beats a generic bigger model, and the entry price for “I trained my own model on my own machine” is a used GPU at $273–$295 fair asking on eBay, August 2026.

The 24GB tier hits a wall at 32B — and 16-bit LoRA is off the menu entirely. A used RTX 3090 ($1,202–$1,296 fair asking range on eBay, August 2026) or an RTX 4090 runs 27B–32B models all day, but training a 32B wants 26GB — 2GB more than the card has. Even 16-bit LoRA on a mere 8B wants 22GB, which fits but leaves nothing. The 24GB sweet spot for inference is a 14B-QLoRA card for training. Full math on whether owning the card beats renting for training workloads is in our QLoRA RTX 4090 vs RunPod cost breakdown.

32B/70B tuning is a rental. 41GB for a 70B QLoRA means dual 24GB cards with all the multi-GPU complexity that entails, or an hour-billed A100 80GB on RunPod — and Unsloth Studio’s headless mode exists for exactly that: same UI, rented silicon.

Unsloth’s headline efficiency claims — training up to 2× faster with up to 70% less VRAM than standard Hugging Face pipelines, and 80% less VRAM for GRPO reinforcement learning — are the reason these floors are as low as they are. The 2×/70% figures come from Unsloth’s published Llama-class benchmarks and are the best case, not the guarantee.

The OOM you will actually hit

The floors above assume short context, batch size 1–2, and a well-behaved dataset. The failure mode when those assumptions break is instructive. From unsloth issue #4504 (March 2026, still open): a user fine-tuning Qwen3.5-2B — a 2-billion-parameter model — with 4-bit QLoRA at 4096 context on an RTX 4070 Ti 12GB got:

CUDA out of memory. Tried to allocate 7.58 GiB. GPU 0 has a total
capacity of 11.56 GiB of which 4.07 GiB is free.

Same GPU, same model, same settings: an Alpaca-style dataset (flat prompt/response columns) trained for thousands of rows, while a ChatML-style dataset (nested role/content messages) blew up on 250 rows. Tokenized chat templates can inflate sequence memory dramatically, and a 2B model OOMing a 12GB card is what that looks like.

The practical defenses, in order: cut max_seq_length to what your data actually needs (2048 covers most instruction datasets); keep batch size at 1–2 and use gradient accumulation for effective batch; prefer flat prompt/response formats where your task allows; and use v0.1.702’s tunable VRAM setting to cap the app’s appetite. If it still OOMs, your card is one tier too small for the job — the table above, plus ~30–40% headroom over the floor, is the honest sizing rule.

Diffusion: real, but ComfyUI keeps its job

Image generation arrived in the Aug 10 builds as “preliminary,” got improvements on Aug 13, and leans on Unsloth’s own GGUF quants. The numbers make the case for small cards: FLUX 2 Klein (the 4B distilled variant) wants ~13GB of VRAM in its native precision, but Unsloth’s GGUF build is 2.6GB at Q4_K_M and 4.3GB at Q8_0, renders in 4 inference steps, and fits an 8GB card even at the higher quant.

What you don’t get is a workflow graph. ComfyUI’s node system — custom LoRA chains, ControlNet, upscalers, video pipelines — has no equivalent here; Unsloth Desktop gives you prompt-in, image-out plus the ability to train diffusion LoRAs in the same app. If your image work is one-shot generation, this is now the simpler tool. If you’ve built actual ComfyUI workflows, nothing changes.

Platform support: the matrix that decides for you

• NVIDIA (CUDA): the first-class path — RTX 30/40/50 series through DGX Spark, run + train, multi-GPU training supported.
• AMD (ROCm/Vulkan): run + train on the ROCm path; RDNA 3/4 and Strix Halo support was rough at launch (GPU detection was broken until v0.1.701) and got explicit fixes in v0.1.800. Works, expect beta friction — same story as AMD local AI generally.
• Intel Arc (Vulkan): inference works via the Vulkan backend.
• Apple Silicon (Metal): chat/inference only today — MLX training is “coming very soon” per the README, so Mac owners get the run half but not yet the half that makes this app special.
• CPU-only: chat inference only.

FAQ

Is Unsloth Desktop free? Yes — free and open source (the repo is Apache 2.0), Windows/macOS/Linux. Unsloth monetizes enterprise, not the desktop app.

Does it replace Ollama? Functionally it can — llama.cpp backend, OpenAI-compatible API, model downloads. Practically, Ollama’s ecosystem (every tutorial, every integration, ollama pull) is the default for serving, and there’s no speed reason to migrate. Run both; they don’t conflict.

Can I fine-tune on 8GB of VRAM? Yes — 7B/8B-class models at QLoRA 4-bit (~6GB floor), short context, batch 1. Keep the desktop environment light; you have maybe 1.5GB of slack.

Can my 24GB card fine-tune the 27B models it runs? Marginally at best. The documented 32B QLoRA floor is 26GB; 27B-class sits right at the card’s edge, and real context lengths push past it. Plan on 14B for comfortable local tuning, or rent for bigger.

Where do trained models go? Export to GGUF or 16-bit safetensors from the same UI — so the output drops straight into Ollama, llama.cpp, or whatever your VRAM tier runs best.

What’s Unsloth Studio vs Unsloth Desktop? Same interface, two packagings: Desktop is the installed app, Studio is the self-hosted web UI (unsloth studio -p 8888) for headless boxes and cloud GPUs.

The verdict

Unsloth Desktop is the first local AI app whose pitch isn’t “run models, but friendlier.” Inference is deliberately boring — llama.cpp with a good UI — and the diffusion side is a convenience, not a ComfyUI threat. The training side is genuinely new territory for a desktop app, and the VRAM floors are low enough to matter: 6GB tunes an 8B, 8.5GB tunes a 14B, and the sub-$300 used-GPU tier just became a fine-tuning tier. Install it alongside your existing stack, not instead of it — and if the model you want to tune is 32B+, the app’s own headless mode on rented silicon is the move.

Sources

• Unsloth repository and README — GitHub
• Unsloth release notes v0.1.61 through v0.1.800-beta — GitHub Releases
• Unsloth Requirements (QLoRA/LoRA VRAM floors) — Unsloth Documentation
• Introducing Unsloth Studio — Unsloth Documentation
• Fine-tuning OOM report, Qwen3.5-2B on RTX 4070 Ti — unsloth issue #4504
• llama.cpp vs Ollama token-generation benchmark — BSWEN
• Image generation VRAM requirements 2026, FLUX 2 Klein GGUF sizes — Will It Run AI
• Used RTX 3090 fair asking range, August 2026 — ResalePrices
• Used RTX 3060 12GB fair asking range, August 2026 — ResalePrices
• Unsloth LICENSE (Apache 2.0) — GitHub
• Unsloth Studio deployment reference — Vast.ai Documentation

Last updated August 16, 2026. Prices and specs change; verify current rates before purchasing.

Recommended Gear

Products linked in this guide:

• RTX 3060 12GB — the cheapest comfortable 14B QLoRA card
• RTX 3090 — 24GB used-value pick: runs 27B, tunes 14B
• RTX 4090 — same 24GB training ceiling, roughly 2× the speed