TL;DR: LLM token generation is memory-bandwidth-bound, not core-clock-bound — so you can cap an RTX 3090 from 350W down to 250W and lose under 3% of your tokens/sec, and cap an RTX 4090 to 70% power for ~94% of its performance. One nvidia-smi -pl command, reversible, no BIOS flash. The two catches: the setting resets on reboot unless you wire up a systemd unit, and there’s a hard cliff below ~250W on the 3090 where the core clock collapses and speed drops 35%.

What you’ll be able to do:

• Set a power cap that survives reboots and shaves roughly $4–14/month per GPU off your electricity bill, depending on duty cycle
• Find your specific card’s sweet spot with a 10-minute test instead of guessing
• Spot (and fix) GDDR6X memory-temperature throttling — the silent slowdown nvidia-smi can’t even see

Honest take: If you run a 24/7 local AI box on an RTX 3090, sudo nvidia-smi -pl 280 is the single highest-value command you’re not running. It costs under 5% of your tokens/sec, drops heat, noise, and PSU stress, and takes ten seconds. Undervolting proper squeezes out a bit more, but the power limit gets you 90% of the benefit with 1% of the fiddling.

Every article on this site that mentions a used RTX 3090 quotes its 350W board power. What most guides skip: that 350W buys you almost nothing during token generation. Decode speed is set by how fast the card can stream weights out of VRAM — the 3090’s 936 GB/s of bandwidth — not by how hard the core is boosting. The card burns 350W chasing core clocks that the workload can’t use.

That’s the whole trick. Cap the power, the core clock drops, the memory clock doesn’t, and your tokens/sec barely moves. We covered why decode is bandwidth-bound in Why Local LLMs Got Good in 2026; this is the guide to cashing that fact in.

The measured numbers: where the sweet spot is

The cleanest public single-card test comes from Jean Brito, who stepped an RTX 3090 through six power limits running Qwen 3.6 27B (dense) in a long-form generation benchmark (4,000 output tokens). The shape of the curve is the story:

Power limit	What happens	Decode speed
350W (stock)	Core boosts to ~1710 MHz	Baseline
300W	Core eases down, memory clock untouched	Within 3% of stock
275W	Still flat	Within 3% of stock
250W	Core ~1395 MHz — the sweet spot	31.7 tok/s (<3% below stock)
200W	The cliff. Core collapses to ~480 MHz	20.6 tok/s (−35%)
150W	Unusable	8.3 tok/s

From 350W to 250W the card gives up less than 3% of its speed for a 100W haircut. Below 250W the card can no longer hold anything close to its base clock, the core craters by 65%, and even a bandwidth-bound workload starves — memory bandwidth doesn’t help if the core is too slow to issue the work.

Three other data points bracket that result from different angles:

• Puget Systems ran quad-3090 machine-learning workloads at 280W per card and measured over 95% of maximum performance — and that was on training jobs, which are compute-bound and should be the worst case for power capping. Their motivation was practical: four 3090s at stock is 1,400W of GPU before the CPU gets a watt, which is wall-circuit territory (a problem we hit again in the $50K quad-RTX-PRO-6000 build).
• Himesh Prasad’s 4× RTX 3090 vLLM benchmarks (March 2025) put the efficiency sweet spot — most tokens per joule under batch serving — around 220W per card, with NVLink adding ~50% tensor-parallel throughput on a 2×3090 pair. If you’re batch-serving overnight jobs rather than chatting interactively, you can go deeper than 250W and come out ahead on tokens per dollar.
• On the RTX 4090, Tom’s Hardware found the 70% power limit (~315W of its 450W stock) delivers ~94% of performance at 75% of the power, and TechPowerUp reported the card losing just 8% with the limit nearly halved plus an undervolt. Community inference reports land in the same band: 50–70% caps keep over 90% of tokens/sec, because — same story — decode never needed the boost clocks.

One honest asymmetry to know before you pick a number: prompt processing (prefill) is compute-bound, unlike decode. If your workload is RAG over huge contexts or agent loops that re-read 30K-token prompts, a deep power cap costs you more in time-to-first-token than in generation speed. Interactive chat with short prompts barely notices; heavy-prefill workloads should stay at the shallow end (280W+ on a 3090, 70–80% on a 4090).

Set it: one command

Check what your specific card allows first — min/max limits vary by AIB model:

$ nvidia-smi -q -d POWER | grep -i "power limit"
        Default Power Limit               : 350.00 W
        Min Power Limit                   : ...
        Max Power Limit                   : ...

The default is 350W on a reference 3090; the min and max vary by AIB model — whatever yours prints is the range the driver will accept (-pl values outside it are rejected with an error, not applied).

Then set the cap (persists until reboot):

$ sudo nvidia-smi -pl 280
Power limit for GPU 00000000:01:00.0 was set to 280.00 W from 350.00 W.

That’s it. No reboot, no driver reload — it applies mid-inference if a model is loaded. Watch the effect live in a second terminal with watch -n1 nvidia-smi while you run your usual ollama run or llama-bench workload: power draw pins at the new cap, the core clock floats down, and your tokens/sec should barely move.

Find your own sweet spot in 10 minutes: run the same fixed prompt at 350W, 300W, 280W, 250W, and 225W, noting tokens/sec each time (Ollama: add --verbose; llama.cpp: llama-bench). Your card, your quant, and your model’s active-parameter count all shift the curve a little. MoE models like Qwen3.6-35B-A3B — which read only ~3B of weights per token — are even more forgiving of deep caps than the dense 27B in the benchmark above.

If you have multiple GPUs, target them individually with -i:

$ sudo nvidia-smi -i 0 -pl 280 && sudo nvidia-smi -i 1 -pl 280

Make it survive reboots (the step everyone skips)

Here’s the problem you’ll actually hit: the power limit silently resets to stock on every reboot. You set 280W in July, a kernel update reboots the box in August, and your “quiet efficient server” is back at 350W without a word. Nothing in journalctl flags it — the only tell is the wattage in nvidia-smi.

The fix on Linux is a oneshot systemd unit. Enable persistence mode too, so the driver holds settings while no process is using the GPU:

$ sudo tee /etc/systemd/system/nv-power-limit.service <<'EOF'
[Unit]
Description=Set NVIDIA GPU power limit
After=syslog.target systemd-modules-load.service

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -pm 1
ExecStart=/usr/bin/nvidia-smi -pl 280

[Install]
WantedBy=multi-user.target
EOF
$ sudo systemctl daemon-reload && sudo systemctl enable --now nv-power-limit.service

If you’d rather not hand-write the unit, Puget Systems maintains an interactive setup script (dbkinghorn/nv-gpu-powerlimit-setup) that detects your GPUs, prompts for a wattage per card, and installs the config (/usr/local/etc/nv-powerlimit.conf), helper script, and service unit for you.

On Windows, nvidia-smi -pl works from an elevated prompt (nvidia-smi ships with the driver, in C:\Windows\System32), but the reboot-reset problem is the same — schedule it via Task Scheduler at logon, or use MSI Afterburner’s power-limit slider with “apply at startup,” which is the usual route since Afterburner also unlocks the voltage-curve editor.

Undervolting proper: worth it on Linux in 2026?

A power limit is a ceiling; an undervolt moves the whole efficiency curve. On Windows, Afterburner’s curve editor makes this a 20-minute job. On Linux, NVIDIA still exposes no direct voltage control — but there’s a well-established workaround: shift the voltage/frequency curve with a positive core-clock offset, then lock the maximum graphics clock (nvidia-smi -lgc) so the card reaches that clock at a lower voltage point than stock. Same clocks, less voltage, less heat.

The nixguru/gpu-undervolt tool scripts exactly this technique (tested on an RTX 3090 with the 580.xx driver; Ampere and newer). A one-shot example from its docs:

sudo gpu_undervolt.py --mode oneshot --index 0 --use-offsets \
  --display :0 --core-offset 150 --memory-offset 500 \
  --min-clock 210 --target-clock 1860 --verify

Its daemon mode applies the undervolt under load and reverts at idle, so an aggressive offset isn’t live while you’re just browsing.

Should you bother? If your box is a headless inference server, mostly no — the flat part of the power-limit curve already captured the easy watts, and an unstable undervolt shows up as the kind of maddening intermittent gibberish we triaged in the repeating/gibberish fix guide. Undervolting earns its keep when you’re thermally constrained (SFF case, hot room, stacked cards) and need to cut heat without giving up the last 3–5% of clocks. Test stability with an hour of sustained generation, not a 30-second benchmark.

The trap: your VRAM can throttle while your GPU temp looks fine

Here’s the real problem-and-fix this article exists for. A reader-familiar scenario: 3090 core temp reads a comfortable 65°C, but tokens/sec sags 10–20% after a few minutes of sustained generation, and the fans spin like jet engines. The cause is usually invisible: GDDR6X memory junction temperature.

The memory chips on the RTX 3090 (and 3080/3090 Ti) run far hotter than the die — 96–108°C under sustained load is the normal range on stock cards — and GDDR6X thermally throttles at 110°C (spec ceiling 120°C). Sustained LLM inference is close to a worst case: it hammers VRAM continuously, and on the 3090 specifically, half the memory chips sit on the back of the PCB with minimal cooling. Early-2021 crypto miners discovered this throttle; 2026 home-lab LLM rigs rediscover it weekly.

The nasty part: on Linux, nvidia-smi cannot show you this number. NVIDIA has never exposed memory-junction temperature through NVML on Linux — there’s a developer-forum request thread that’s been open for years. Your monitoring says 65°C while the VRAM quietly clamps.

How to actually read it:

• Windows: HWiNFO64 has shown “GPU Memory Junction Temperature” since v6.42.
• Linux: the open-source olealgoritme/gddr6 tool reads the sensor directly over PCIe (reverse-engineered). It needs iomem=relaxed on the kernel command line, Secure Boot disabled, and libpci-dev installed; then sudo gddr6 prints the hottest module. It covers RTX 3000/4000 series and professional cards, with experimental GDDR7/RTX 5090 support.

And the fixes, in order of effort:

1. Power-limit the card (you were doing this anyway) — less board power means cooler memory, and at 250–280W many 3090s drop below the throttle point with no hardware work.
2. Raise case airflow past the backplate. The rear memory chips cool through the backplate; a cheap fan aimed at it measurably helps.
3. Replace the thermal pads. Stock pads on many 3090s are mediocre; the well-documented mod replaces them with higher-conductivity aftermarket pads and drops VRAM temps by up to 25°C (Notebookcheck covered a mod hitting exactly that). It voids some warranties — largely moot on a used card in 2026 — and requires a full teardown, so it’s the last resort, not the first.

If you bought one of the ~$1,050 used 3090s (eBay floor, August 2026 — market average $1,248 across 374 listings per ResalePrices) that this site keeps recommending, assume the pads are five years old and the card mined for part of its life. Twenty minutes with gddr6 running during a long generation tells you whether you got a good one.

What the power bill math looks like

Our math, flagged as such, at the EIA’s 18.83¢/kWh US residential average (April 2026): a 100W cut (350W→250W on a 3090) saves 2.4 kWh/day on a GPU pegged 24/7 — about $13.50/month, $162/year, per card. At a more typical 8 hours/day of actual load, it’s ~$4.50/month. A quad-3090 rig capped at 280W each saves ~280W total — roughly $38/month if the rig runs hard around the clock, plus a quieter room and a PSU running far from its redline (see PSU sizing for AI workstations for why that headroom matters).

Two things a power limit does not do: it doesn’t touch idle draw (a 3090 idles around 20W regardless — capping helps loaded hours only), and it doesn’t shrink the up-front hardware cost. If your real constraint is a 15A circuit that can’t feed a multi-GPU rig at any power limit, renting the occasional big job on RunPod (A100 80GB from $1.39/hr) beats tripping breakers.

For the full cost-of-ownership picture — idle floors, duty cycles, and when a home server beats the cloud — see the 24/7 AI server power-bill math, which this guide expands from a single paragraph into practice.

FAQ

Can power limiting damage my GPU or void the warranty? No. nvidia-smi -pl is a driver-level cap within the vendor’s allowed range (the driver rejects values outside Min/Max Power Limit), and it’s fully reversible with a reboot or nvidia-smi -pl 350. It’s the opposite of overclocking: everything runs cooler and further inside spec. Physical thermal-pad replacement is the only thing in this guide with warranty implications.

Do MoE models respond differently than dense models? They tolerate deeper caps. A 35B-A3B MoE reads ~3B parameters per token, so it demands even less core throughput per unit of bandwidth than a dense 27B — the flat part of the curve extends further left. Dense models at long context, and anything prefill-heavy, hit the wall soonest.

Power limit or undervolt — which first? Power limit first, always: one command, zero stability risk, captures most of the savings. Undervolt on top only if you’re thermally constrained or chasing the last few percent of efficiency, and stability-test for at least an hour of sustained generation.

Does this work on AMD cards? The same physics applies — decode is bandwidth-bound on an RX 7900 XTX too — and AMD exposes power caps through rocm-smi. The measured curves in this article are NVIDIA-specific, though; test your own sweet spot before committing a number to a service file.

What about the RTX 5090 and its 575W board power? Same logic, bigger prize — there’s more waste heat to claw back at stock, and 5090 owners report large caps with minor decode loss. We haven’t found a stepped, published 5090 LLM power-scaling curve we’d cite yet; run the 10-minute self-test above and trust your own numbers. (The gddr6 tool’s GDDR7 readings on the 5090 are experimental — treat them as approximate.)

Recommended Gear

• RTX 3090 — still the bandwidth-per-dollar king at ~$1,050 used; cap it at 280W and it’s also quiet
• RTX 4090 — at a 70% power limit it delivers ~94% performance at 315W

Sources

• RTX 3090 Power Limit: Finding the Sweet Spot for Local LLM Inference — Jean Brito
• Quad RTX3090 GPU Power Limiting with Systemd and Nvidia-smi — Puget Systems
• nv-gpu-powerlimit-setup (systemd setup script) — dbkinghorn/Puget Systems, GitHub
• vLLM Performance Benchmarks 4x RTX 3090, Power Limits and NVLink — Himesh Prasad
• Improving Nvidia RTX 4090 Efficiency Through Power Limiting — Tom’s Hardware
• RTX 4090 with Nearly Half its Power Limit and Undervolting Loses Just 8% Performance — TechPowerUp
• GDDR6X Memory Temperature: Why 110 Degrees Is Normal and When to Worry — VoltGround
• Request: GPU Memory Junction Temperature via nvidia-smi or NVML API — NVIDIA Developer Forums
• gddr6: Linux GDDR6/GDDR6X VRAM temperature reader — olealgoritme, GitHub
• gpu-undervolt: Linux GPU Undervolt Tool for Modern NVIDIA GPUs — nixguru, GitHub
• RTX 3090 mod can reduce VRAM temperatures by up to 25°C — Notebookcheck
• RTX 3090 Price Tracker US, August 2026 — Best Value GPU
• RTX 3090 Used GPU Price and Fair Asking Range — ResalePrices
• Electric Power Monthly, average residential price — U.S. Energy Information Administration

Last updated August 5, 2026. Prices, drivers, and specs change; verify current rates before purchasing. If you’re wiring a local model into your coding editor, our sister site aicoderscope.com covers the tool side.