CUDA Error: No Kernel Image Is Available? Fix the PyTorch sm_120 and Old-GPU Mismatch (2026)

By RunAIHome Team · 12 min read
pytorchcudartx-5090troubleshootinggpulocal-ai

TL;DR: RuntimeError: CUDA error: no kernel image is available for execution on the device means your PyTorch wheel ships no compiled code for your GPU’s compute capability. It hits two opposite groups: RTX 50-series owners on pre-2.7 PyTorch (Blackwell needs sm_120, first stable in PyTorch 2.7.0 + CUDA 12.8), and GTX 10-series/older owners on PyTorch 2.8+, which dropped Maxwell, Pascal, and Volta kernels. One diagnostic command tells you which side you’re on.

What you’ll be able to do:

  • Read torch.cuda.get_arch_list() and know in 60 seconds whether your wheel, your GPU, or a compiled extension is the problem
  • Fix an RTX 5090/5080/5070/5060 by installing the right wheel index (cu130 in August 2026 — not the cu128 that 2025 guides still recommend)
  • Keep a GTX 10-series or Tesla P40 working by pinning the last wheels that still contain its kernels

Honest take: This error is almost never a bug. Since April 2025 the fix for Blackwell cards has been a one-line reinstall, and the 2026 cases are stale virtual environments and requirements files pinning 2024-era wheels. Diagnose first — the rebuild-from-source advice all over old forum threads is the last resort, not the first.

The cruelest part of this error is that everything looks fine right up to the crash. nvidia-smi shows your card. torch.cuda.is_available() returns True. The model loads. Then the first actual CUDA operation — a matmul, a sampler step in ComfyUI, a vLLM warmup pass — throws:

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

is_available() only checks that a driver and a CUDA device exist. It does not check that your PyTorch build contains machine code for that device’s architecture. When it doesn’t, the GPU is detected, addressed, and completely unusable — like having the right address but a key cut for a different lock.

What the error actually means

Every CUDA binary (a “kernel image”) is compiled for a specific GPU architecture, named by compute capability: the RTX 3090 is 8.6 (sm_86), the RTX 4090 is 8.9 (sm_89), and every RTX 50-series card from the 5060 to the RTX 5090 is 12.0 (sm_120) on the Blackwell architecture (NVIDIA forums).

To keep download sizes sane, PyTorch wheels bundle kernels for a fixed list of architectures. If your card’s compute capability isn’t covered by that list, every kernel launch fails with this exact error. Two events created today’s two failure classes:

  1. Blackwell arrived (early 2025). sm_120 kernels first shipped in stable PyTorch 2.7.0 with CUDA 12.8 wheels, released April 2025 (PyTorch 2.7 release blog, pytorch#159207). Anything older — including the torch 2.4–2.6 pins still baked into countless requirements.txt files — cannot drive an RTX 50-series card.
  2. Pre-Turing left (mid 2025). NVIDIA removed Maxwell, Pascal, and Volta (compute 5.x–7.0) from CUDA 13.0 offline compilation (Tom’s Hardware, CUDA 13.0 release notes), and PyTorch dropped those kernels from its CUDA 12.8/12.9 builds starting with release 2.8 (pytorch#157517, dev-discuss announcement). A GTX 1080 Ti that ran fine on torch 2.7 throws this error the day you upgrade.

The wheel arch list for current builds is 7.5; 8.0; 8.6; 9.0; 10.0; 12.0 — Turing and newer. Note what’s missing on both ends.

The 60-second diagnosis

Run this in the same Python environment that’s crashing (activate the venv first — testing the wrong environment is the #1 way this diagnosis goes sideways):

python -c "import torch; print(torch.__version__); print(torch.cuda.get_device_capability()); print(torch.cuda.get_arch_list())"

Healthy output on an RTX 5090 with a current wheel looks like:

2.13.0+cu130
(12, 0)
['sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']

Read it as two numbers and one list:

  • Line 2 is your GPU. (12, 0) = sm_120 (any RTX 50 card), (8, 9) = RTX 4090, (8, 6) = RTX 3090, (6, 1) = GTX 10-series.
  • Line 3 is what your wheel can actually run. If your GPU’s architecture (or a lower minor version within the same major number — more on that below) is not in this list, you’ve found the problem.
  • Line 1 tells you why. A +cu121 or +cu124 suffix, or a version below 2.7.0, on a Blackwell card = failure class 1. A version of 2.8+ on a Pascal card = failure class 2.

One nuance that stops false alarms: kernels are forward-compatible within the same major compute capability. An RTX 4090 (8.9) runs the sm_86 kernels happily, which is why sm_89 doesn’t appear in the list and nothing is wrong (PyTorch forums). That compatibility does not cross major versions: nothing in sm_86–sm_90 will ever run on a 12.0 card.

If the list does contain your architecture and you still get the error, skip to Fix C — the broken kernel image belongs to a compiled extension, not to PyTorch itself.

Fix A: RTX 50-series card, wheel too old

This is the overwhelming majority of cases, and it’s a reinstall, not a rebuild. Inside your project’s venv:

pip uninstall -y torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130

Then re-run the diagnostic — you want sm_120 in the list.

Three traps worth knowing in August 2026:

  • The 2025 guides say cu128. The matrix moved. PyTorch 2.13.0 (current stable, July 2026) removed CUDA 12.8 from its binary build matrix: CUDA 13.0 (cu130) is now the stable default, with cu126 kept only for older drivers (PyTorch releases, pytorch#172926). Pinning the cu128 index still works — it just silently caps you at torch 2.12.
  • cu130 needs a current driver. CUDA 13 runtime wants the R580+ driver branch (flash-attention#1638). If nvidia-smi shows an older driver and you can’t upgrade it, use --index-url https://download.pytorch.org/whl/cu128 with torch==2.12.* instead — those wheels carry sm_120 too. (If nvidia-smi itself errors, that’s a different problem with its own fix.)
  • cu126 will not save a Blackwell card. CUDA 12.6 predates sm_120 entirely. On RTX 50-series, the “older drivers” fallback index is a dead end.

The pattern that actually generates these failures in 2026 is stale environments. A representative report from January 2026 (pytorch#173237) shows an RTX 5090 owner crashing on 2.7.0.dev20250310+cu124 — a nightly snapshot from March 2025, ten months old, frozen in a venv that predated stable Blackwell support. The maintainers closed it as not-planned because the fix has existed since 2.7.0. Before filing anything, check what your requirements.txt, conda env, or Docker base image is actually pinning. Modern installers can bite here too: uv’s --torch-backend=auto has picked wheels without checking compute capability at all (uv#14742).

Fix B: GTX 10-series or older, wheel too new

If the diagnostic shows (6, 1) (GTX 1080/1070/1060, Tesla P40), (5, 2) (GTX 900), or (7, 0) (Tesla V100, Titan V) and torch 2.8+, your card’s kernels were removed from the wheels you installed. Two working options:

Option 1 — pin the last fully-supported release:

pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu126

PyTorch 2.7.x is the last release line where every wheel variant still carried Pascal kernels; community consensus on the forums is blunt about it — “you need to use PyTorch 2.7 or earlier” for a 1080 Ti (PyTorch forums).

Option 2 — stay current but use the cu126 index, which retained pre-Turing kernels after the cu128/cu129 builds dropped them (dev-discuss). Verify with get_arch_list() after install rather than trusting it — coverage has not been uniform across platforms (a Windows 2.8.0+cu126 build shipped without Maxwell kernels, pytorch#160575).

The honest strategic advice: treat this as the retirement bell for Pascal in PyTorch stacks. If your old card is mainly for LLM inference, llama.cpp and Ollama build their own CUDA code and still target Pascal — a P40 that’s useless under torch 2.13 keeps serving GGUF models fine (see our llama.cpp CUDA build guide). For occasional PyTorch work that genuinely needs a modern stack, renting a current GPU on RunPod for $1–3/hour beats fighting an architecture NVIDIA has ended; and if you’re upgrading instead, a used RTX 3090 (compute 8.6, safely mid-list) remains the value pick — see why it’s still the value king.

Fix C: PyTorch is fine — a compiled extension isn’t

If get_arch_list() shows your architecture and the error persists, the failing kernel image belongs to a separately-compiled CUDA extension: flash-attention, bitsandbytes, xformers, a ComfyUI custom node with its own ops, an exllama kernel. These compile against whatever architecture list existed at their build time, so a wheel or cached build from before your GPU (or your torch upgrade) fails the same way PyTorch itself would.

The traceback usually names the culprit (flash_attn, bitsandbytes, a custom_nodes/ path). Rebuild it for your architecture — for flash-attention on an RTX 50-series card (flash-attention#1638, ai-toolkit#475):

pip uninstall -y flash-attn
export TORCH_CUDA_ARCH_LIST="12.0"
export FORCE_CUDA=1
pip install ninja packaging
pip install flash-attn --no-build-isolation

TORCH_CUDA_ARCH_LIST is the lever: it tells the extension’s build exactly which kernel images to produce. Set it to your card’s compute capability ("8.6" for a 3090, "8.9" for a 4090, "12.0" for Blackwell). The same recipe works for most pip-installed CUDA extensions; for packages that ship prebuilt wheels (bitsandbytes, xformers), first just upgrade — current releases include sm_120 images.

Where you’ll meet this error, tool by tool

ToolTypical triggerFastest fix
ComfyUI (portable)Custom node’s pip install downgraded torch, or old bundled wheel on a new cardReinstall torch inside python_embeded — full walkthrough
vLLMHistorical on Blackwell (fixed since mid-2025, vllm#16901); now mostly stale venvsUpgrade vLLM (current wheels build on CUDA 13) — vLLM startup errors guide
Ollama / LM StudioDoesn’t use your Python torch at allIf these error, it’s not this bug — check the llama.cpp build guide for the compute_120 equivalent
Training scripts (LoRA, ai-toolkit)flash-attn/bitsandbytes compiled for the wrong archFix C rebuild with TORCH_CUDA_ARCH_LIST
Docker imagesBase image frozen with pre-2.7 torchRebuild from a current base; GPU passthrough itself is a separate checklist

If you’re wiring a local model into a coding agent and hitting this mid-setup, our sister site aicoderscope.com covers the tool side of local backends for Cursor, Cline, and Claude Code.

FAQ

torch.cuda.is_available() returns True — how can the GPU be unsupported? is_available() verifies driver + device presence, not kernel coverage. Architecture support only gets tested when a kernel actually launches. That’s why the crash arrives at the first real operation instead of at import.

My 4090 shows no sm_89 in the arch list. Is that a problem? No. Compiled kernels run on any GPU with the same major compute capability and an equal-or-higher minor version, so the sm_86 and sm_80 images cover 8.9 (PyTorch forums). Only a missing major architecture (no sm_120 on a 12.0 card) means failure.

Do I need to install the CUDA Toolkit to fix this? No. PyTorch wheels bundle their own CUDA runtime; the only system component that matters is the NVIDIA driver. Installing or reinstalling the toolkit changes nothing about a wheel’s compiled arch list. The toolkit only enters the picture in Fix C, where an extension compiles against it.

Why did it work last week and break today? Something changed the wheel under you: a pip install from a custom node or requirements file that downgraded torch, a Docker image rebuild, or an upgrade past 2.8 on a pre-Turing card. The GPU didn’t change; the kernel list did. Re-run the diagnostic and compare against what the environment had before.

Is a used RTX 3090 safe from this whole mess? For years, yes. Compute 8.6 sits comfortably inside every current wheel’s list, and Ampere is far from NVIDIA’s deprecation queue — Turing (7.5, one generation older) is still shipping in cu130 wheels. It’s one more argument in the used 3090 value case.

  • NVIDIA RTX 5090 — 32GB Blackwell flagship; needs torch ≥2.7 (cu128) or current cu130 wheels
  • NVIDIA RTX 3090 — used 24GB value pick, compute 8.6, supported by every current wheel

Sources

Last updated August 7, 2026. Wheel indexes, driver requirements, and version support move fast; verify against the current PyTorch install matrix before pinning.

Was this article helpful?