LM Studio Not Using Your GPU? Fix the Offload Slider, CPU Spillover, and Slow Tokens/Sec (2026)

By RunAIHome Team · 11 min read
lm-studiolocal-llmtroubleshootinggpuai

TL;DR: If LM Studio is crawling at single-digit tokens/sec, your model is running on the CPU — not the GPU. In almost every case it’s one of three things: the GPU Offload slider is set below “max,” the wrong runtime engine is installed (or none is), or your graphics driver is more than a year old and the GPU never gets detected. Fix those in order and you go from ~4 tok/s to 35–76 tok/s on the same card.

What you’ll be able to do after this article:

  • Confirm in ten seconds whether your model is actually on the GPU (the lms ps check)
  • Install the correct runtime engine — CUDA, Vulkan, or MLX — and max out the offload slider without triggering an out-of-memory crash
  • Diagnose the harder cases: partial spillover, KV-cache blowout on long context, integrated-GPU confusion, and multi-GPU priority bugs

If you also run Ollama, the CPU-fallback logic is nearly identical — see Ollama Not Using GPU? Force CUDA Instead of CPU.

First, confirm the problem is real

“Slow” is not a diagnosis. Before changing anything, prove where the model is running. Load your model, then open a terminal and run the LM Studio CLI:

lms ps

Output looks like this:

LOADED MODELS
Identifier: qwen2.5-7b-instruct
  Status: LOADED
  GPU: 0/33 layers    ← the number that matters
  Context: 4096

GPU: 0/33 layers means zero transformer layers are on the GPU. Every token is being computed on your CPU. 33/33 means the whole model is resident on the GPU — that’s what you want. Anything in between (say 18/33) is a hybrid split, and hybrid is slow because the CPU half becomes the bottleneck for every single token.

The performance gap is not subtle. On a mid-range RTX 3060, the same 7B model measures roughly:

Offload settingWhere it runsTokens/sec (7B Q4)
0 layersCPU only~4 tok/s
~18/33 layersHybrid (spillover)~22 tok/s
33/33 layersFull GPU~35 tok/s

A fully GPU-resident model produces roughly 3–10× more tokens per second than CPU-only inference. On a 24 GB RTX 3090 — 936 GB/s of memory bandwidth — a Q4 7B model at full offload benchmarks around 76 tok/s. If you’re seeing 4, the card isn’t the problem. The configuration is.

Fix 1: Max out the GPU Offload slider

This is the single most common cause, and the search-history data backs it up: people report their local models as “unusably slow” when the only problem was a slider left at a low value.

In LM Studio, when you load a model (or in the model’s settings panel), there’s a GPU Offload control — it maps directly to llama.cpp’s n_gpu_layers. Drag it to max. If the whole model fits in VRAM, all layers go to the GPU and you get the top row of that table.

How much fits? A rough rule: a 4-bit (Q4_K_M) model needs about 0.6 GB of VRAM per billion parameters, plus headroom for the KV cache and context. A 7B model is ~5 GB; a 13B is ~8 GB; a 34B needs ~20 GB. If your card has the room, offload everything. If it doesn’t, that’s Fix 4. For the full model-to-VRAM map, see Best Local AI Models by VRAM Tier, and if the quant names confuse you, Local LLM Quantization Explained untangles Q4 vs Q5 vs Q8.

Fix 2: The slider is grayed out or stuck at 0 — install the runtime

If the GPU Offload slider is grayed out, capped at 0, or the load dialog says “GPU offload not available,” the app never found a runtime that can talk to your card. This trips up a lot of new users because of one design fact:

LM Studio ships as two separate pieces. There’s the app (the GUI — version 0.4.16 as of June 2026) and there’s the runtime: the actual llama.cpp or MLX engine that loads and executes the GGUF. The app can update itself while the GPU runtime is missing, out of date, or set to a CPU-only build. If the runtime that matches your hardware was never installed, GPU Offload has nothing to act on.

Fix it in Settings → Runtime (or Developer → Runtimes in older builds). Install and select the engine for your GPU:

  • NVIDIA (Windows/Linux) → CUDA llama.cpp. With a compatible driver, LM Studio auto-upgrades to the CUDA 12.8 runtime, which noticeably improved model-load and response times on RTX cards starting in 0.3.15.
  • AMD and Intel → Vulkan llama.cpp. LM Studio is no longer NVIDIA/Apple-only — Vulkan covers most AMD and Intel Arc cards, and some AMD cards on Linux can use ROCm.
  • Apple Silicon (M-series Macs) → MLX. This is the native Apple engine and it’s faster than the llama.cpp Metal path on M-chips.

After selecting the engine, reload the model. The offload slider should now be live. If you’re on a multi-GPU rig, note that 0.4.16 specifically fixed multi-GPU selection bugs affecting GPU ON/OFF and Priority Order on some CUDA 12, ROCm, and Vulkan setups — so if one of your cards was being ignored, updating to 0.4.16 or later may be the entire fix.

Fix 3: The GPU is detected but never used — update the driver

Here’s the sequence that has to succeed for offload to work: driver → runtime engine → CUDA/Vulkan layer → LM Studio. A break anywhere in that chain leaves you on the CPU. The most common break is a stale driver.

A graphics driver more than roughly 12 months old is a frequent cause of failed GPU initialization — the runtime loads, tries to bind to the card, and silently falls back to CPU. Update it:

  • NVIDIA: install the latest Game Ready or Studio driver from NVIDIA. Then confirm the card is visible to the system:
nvidia-smi

If nvidia-smi itself errors with a version mismatch, that’s a driver/library problem underneath LM Studio — fix it first using nvidia-smi “Driver/library version mismatch”.

  • AMD: install the current Adrenalin driver (Windows) or a recent Mesa/ROCm stack (Linux).
  • Intel Arc: install the latest Arc driver — Vulkan support depends on it.

One Linux-specific trap worth calling out: on some integrated-plus-discrete laptops and mini PCs, LM Studio’s Vulkan runtime can bind to the integrated GPU instead of the discrete one, because the iGPU enumerates first. Framework Desktop and similar Ryzen AI systems have hit this. If lms ps shows layers “on GPU” but performance is still terrible, check which GPU — the iGPU has a fraction of the bandwidth.

Fix 4: It works, then spills to CPU mid-generation

You set offload to max, lms ps showed 33/33, the first few messages flew — and then a long chat slowed to a crawl. This is the sneakiest case, and it’s a VRAM problem, not a config problem.

The model weights fit, but the KV cache grows with context. Every token in the conversation consumes VRAM, and LM Studio’s memory estimator is a beta feature that doesn’t fully account for KV-cache growth during long generations. So you load fine at 512 tokens and blow past your VRAM ceiling at 8,000 — at which point layers get evicted to system RAM and you’re back to hybrid speed. (LM Studio 0.4.7 fixed some of the inaccurate guardrail estimates, but the underlying growth is still real.)

Three fixes, cheapest first:

  1. Lower the context length at load time. If you set 32K “just in case” but your chats are short, drop it to 4096 or 8192. This reclaims a lot of VRAM.
  2. Enable Flash Attention (default on CUDA in recent builds) and, if available, KV-cache quantization to Q8 or Q4 in the load settings — it shrinks the cache substantially.
  3. Drop to a smaller quant. A Q4_K_M instead of Q6 or Q8 frees several GB. The quality loss on most 7B–14B models is minor.

If you keep hitting the ceiling, you’re seeing the same underlying condition as a hard CUDA Out of Memory error — the difference is only whether LM Studio crashes or quietly degrades.

Fix 5: Multi-GPU — the second card sits idle

On a two-card rig you may see one GPU maxed and the other at 0%. LM Studio added real multi-GPU controls in 0.3.14, and 0.4.15 (June 4, 2026) added tensor parallelism for multi-GPU model loading on CUDA, plus a Physical Batch Size advanced option. To use both cards:

  • Open the multi-GPU controls and confirm both GPUs are toggled ON and in the priority order you want.
  • For a model larger than one card’s VRAM, tensor parallelism (0.4.15+) splits it across cards instead of overflowing to CPU.
  • If a card that should be active isn’t listed, you’re almost certainly hitting the pre-0.4.16 selection bug — update.

The 60-second checklist

Run this top to bottom and stop when tokens/sec jumps:

  1. lms ps → is it 0/33? Then it’s on CPU. Continue.
  2. Drag GPU Offload to max. Reload. Re-check lms ps.
  3. Slider grayed out? Settings → Runtime → install CUDA / Vulkan / MLX for your card. Reload.
  4. Still CPU? Update your GPU driver (NVIDIA/AMD/Intel). Confirm with nvidia-smi. Reload.
  5. Fast then slow? Lower context length, enable Flash Attention + KV quant, or drop the quant.
  6. Multi-GPU idle card? Update to 0.4.16+, toggle both GPUs ON, enable tensor parallelism.

When your GPU genuinely can’t run the model

Sometimes the honest answer is that an 8 GB card can’t hold a 32B model no matter how you configure it, and hybrid CPU spillover will always be slow. Two paths:

  • Buy more VRAM. A used RTX 3090 at 24 GB remains the value pick for local inference in 2026 — see Used RTX 3090 in 2026: Still the AI Value King?.
  • Rent a GPU by the hour. For an occasional large model, spinning up a cloud GPU on RunPod is cheaper than buying, and LM Studio’s OpenAI-compatible server can point at it.

If you’re weighing LM Studio against other runtimes entirely, Ollama vs LM Studio vs llama.cpp breaks down which one wins for which use case, and aifoss.dev’s Ollama review goes deeper on the open-source engine side. Using a local model as a coding backend? aicoderscope.com covers wiring LM Studio’s server into Cursor and Cline.

FAQ

Why is LM Studio slow even though lms ps shows some GPU layers? A hybrid split (e.g. 18/33) is bottlenecked by the CPU portion — every token waits on the slowest half. Aim for full offload. If the model won’t fit, lower the quant or context so all layers fit in VRAM.

Do I need to install CUDA separately? No. LM Studio bundles its own CUDA runtime and auto-upgrades to CUDA 12.8 with a compatible driver. You only need a current NVIDIA graphics driver, not the standalone CUDA Toolkit.

LM Studio uses my integrated GPU instead of my real one. How do I fix it? This mostly hits Linux Vulkan setups where the iGPU enumerates first. Update your discrete GPU’s driver, and in multi-GPU controls disable the integrated adapter or set the discrete card as priority.

Does MTP / speculative decoding help GPU throughput? Yes. MTP (Multi-Token Prediction) speculative decoding is stable as of LM Studio 0.4.14 Build 4 and delivers a practical 1.5×–3× throughput gain depending on model and hardware — with no second draft model loaded into VRAM. But it only helps once the model is actually on the GPU; fix offload first.

How do I know how much VRAM a model needs before downloading? Estimate ~0.6 GB per billion parameters for Q4, plus 1–3 GB for context and KV cache. A 13B Q4 model wants roughly 8–10 GB. The Best Local AI Models by VRAM Tier chart lists exact fits per card.

Sources

Last updated July 16, 2026. Software versions, prices, and specs change; verify current rates before purchasing.

  • RTX 3060 — 12 GB budget entry point for 7B–13B models at full offload.
  • RTX 3090 — 24 GB, 936 GB/s; the value king for running 30B-class models fully on the GPU.

Was this article helpful?