TL;DR: A second used RTX 3090 gets you to 48GB for around $2,100–$2,500 (Jul 2026), but on default BIOS settings most dual-3090 rigs run silently below spec — Gen 3 links, P2P traffic bounced through the CPU, and ASPM stutter between prompts. Worse, a badly-wired second slot can produce garbage output above 2048 tokens of context. This guide shows how to check what you actually have and fix it.

What you’ll be able to do after this guide:

• Read nvidia-smi topo -m and tell whether your two cards can actually talk to each other, or are quietly falling back to system-memory staging.
• Fix the three BIOS/kernel defaults — IOMMU/ACS, PCIe link speed, and ASPM — that cap dual-GPU performance and cause the “garbage above 2K context” bug.
• Wire llama.cpp and vLLM correctly so 48GB delivers a real 1.5–2× gain instead of behaving like two separate single-GPU rigs.

Honest take: A correctly-configured pair of used RTX 3090s (48GB, ~$2,100–$2,500 used) is still the cheapest comfortable way to run 70B-class and 27B–35B models fast at home. But “correctly configured” is doing a lot of work in that sentence. Skip the topology check and the BIOS fixes and you’ve spent $1,254 on a card that performs like it isn’t there — or, worse, corrupts long-context output. Spend twenty minutes in the BIOS before you blame the hardware.

Why your second 3090 might be doing nothing

Adding a second RTX 3090 is the classic home-lab upgrade: two 24GB cards give you 48GB of VRAM, enough to hold a 70B model at Q4 or run a 27B–35B model with a huge context window. The used 3090 is still the value pick — market average around $1,254 in July 2026 (fair range roughly $1,194–$1,306 across ~319 tracked listings), with clean single cards occasionally dipping near $1,050. Two of them lands you at 48GB for about the price of one new RTX 5090.

The problem is that a dual-GPU rig has a failure mode a single card doesn’t: the two cards have to move data between themselves, and how fast that happens — or whether it happens correctly at all — depends entirely on how the PCIe fabric is wired and configured. Motherboard defaults are tuned for gaming and virtualization, not for GPU-to-GPU DMA. The result is a rig that boots, shows both cards in nvidia-smi, loads a model across both, and then runs at single-GPU speed while you wonder why the second card didn’t help.

Decode on a local LLM is memory-bandwidth-bound. A single RTX 3090 at 936 GB/s does roughly 95 tok/s on a 7B model. The second card doesn’t make that 7B faster — it lets you run a bigger model that wouldn’t fit on 24GB. That’s the whole point of the upgrade, and it’s exactly the workload that stresses the cross-GPU link. If the link is broken, you don’t just lose speed. You can lose correctness.

Step 1: read your topology before you touch anything

Everything starts here. Run:

nvidia-smi topo -m

You’ll get a matrix. The code connecting GPU0 and GPU1 is the one that matters:

Code	Meaning	Good for LLM split?
NV#	NVLink (# = number of links)	Best — direct GPU-to-GPU
PIX	Single PCIe switch/bridge	Good — P2P over the switch
PXB	Multiple PCIe bridges	OK
PHB	Through the CPU’s PCIe host bridge (root complex)	Works, but P2P may be blocked
SYS	Across the CPU/SMP interconnect (different root complexes)	Slowest — host-staged, no P2P

If you see SYS, your two cards are on different root complexes (common when one slot hangs off the chipset). That means every byte moving between GPUs is copied out to system RAM and back — no direct peer-to-peer DMA. On consumer boards, one GPU on the CPU’s x16 lanes and a second on a chipset x4 slot is the single most common bad layout.

Next, confirm the link speed each card actually negotiated:

nvidia-smi --query-gpu=pcie.link.gen.current,pcie.link.width.current --format=csv

A 3090 wants Gen 4 x16. BIOS “Auto” frequently picks Gen 3, and a chipset slot is often physically wired x4. Gen3 x4 is one-eighth the bandwidth of Gen4 x16 — and that’s where things go wrong.

The bug that proves this matters: garbage output above 2048 tokens

This isn’t theoretical. llama.cpp issue #20052 documents a dual RTX 3090 rig (MSI MAG B550 Tomahawk, Ryzen 5 5600) where GPU 0 sat on the CPU’s Gen4 x16 slot and GPU 1 hung off a chipset Gen3 x4 slot with no P2P. With --split-mode layer and context above 2048 tokens, the model produced “completely incoherent output — repeating ? characters, random tokens, garbled text.”

The cause: with no peer-to-peer path, cross-GPU KV-cache transfers get host-staged through system memory, and the data corrupts once the buffer crosses a size threshold at larger contexts. The reporter’s workarounds were telling — drop context to ≤2048, or fall back to a single GPU. Environment tweaks (GGML_CUDA_DISABLE_GRAPHS=1, GGML_CUDA_PEER_MAX_BATCH_SIZE=0, GGML_CUDA_FORCE_MMQ=1) had no effect. The real fix was the topology, not a flag.

If your dual-3090 setup gives you nonsense on long prompts, this is almost certainly you. Fix the wiring, not the sampler.

Step 2: IOMMU / ACS — the P2P bounce

Here’s the subtle one. Even when both cards sit on CPU-direct lanes, IOMMU (Intel VT-d / AMD-Vi) with ACS (Access Control Services) enabled forces all PCIe peer traffic through the CPU root complex instead of letting it go GPU-to-GPU directly. NVIDIA’s own NCCL troubleshooting docs are explicit: “IO virtualization (also known as VT-d or IOMMU) can interfere with GPU Direct by redirecting all PCI point-to-point traffic to the CPU root complex, causing a significant performance reduction or even a hang.”

Symptom: P2P bandwidth tests show far below the ~20+ GB/s a working PCIe P2P link should hit, or NCCL/vLLM hangs on startup, or dual-GPU speed matches single-GPU. You can confirm with the CUDA samples p2pBandwidthLatencyTest.

Two ways to fix it:

Option A — turn IOMMU off (simplest for a dedicated AI box). If you’re not doing VFIO GPU passthrough to VMs, you don’t need IOMMU. Add to your kernel command line (/etc/default/grub, GRUB_CMDLINE_LINUX_DEFAULT):

amd_iommu=off    # AMD platforms
# or
intel_iommu=off  # Intel platforms

Then sudo update-grub && sudo reboot. Many boards also expose “IOMMU” / “VT-d” / “SVM” directly in BIOS — disable it there.

Option B — keep IOMMU but disable ACS at runtime. If you need IOMMU for other reasons, you can clear the ACS control bits per device so peer traffic isn’t redirected. Find the bus IDs (lspci | grep -i nvidia) and, as root:

sudo setpci -s <BDF> ECAP_ACS+0x6.w=0000

Use the symbolic ECAP_ACS+0x6 form rather than a hardcoded hex offset — the offset differs per board, and writing the wrong register is how you hang a machine. Run it for each PCIe bridge/slot in the path (the GPU’s BDF and the bridge above it). This does not survive reboot, so wrap it in a systemd unit if you want it permanent. This is the same mechanism the pcie_acs_override kernel patch automates, but setpci needs no patched kernel.

Step 3: force the PCIe link speed

If nvidia-smi showed Gen 3 on a Gen 4 board, go into BIOS → Advanced → PCIe settings and set the relevant slot(s) from Auto to Gen 4 explicitly. On some boards “Auto” negotiates down when it sees a riser cable or a second populated slot. Re-check with the --query-gpu=pcie.link.gen.current command above after reboot.

There’s nothing you can do about a slot that’s physically wired x4 — that’s a board limitation. If your second slot is chipset x4, know that layer-split will host-stage through it, and prefer a board with two CPU-direct x8/x8 slots for a serious dual-GPU build.

Step 4: kill ASPM stutter

Active-State Power Management (ASPM) drops the PCIe link into a low-power idle (L0s/L1) between bursts of traffic. On an interactive LLM, that means the link is asleep when your next prompt arrives, adding a cold-start stutter (tens to low-hundreds of milliseconds) to time-to-first-token on each new request. For a chat rig, that’s the difference between “instant” and “why did it pause.”

The clean, safe fix is the kernel command line:

pcie_aspm=off

Add it alongside your IOMMU setting in GRUB, update-grub, reboot. This disables ASPM globally — fine for a plugged-in desktop where you don’t care about a few watts of PCIe idle savings.

You can target a single device with setpci on the Link Control register (CAP_EXP+0x10, where the low two bits select the ASPM state — 00 = disabled), but it’s a read-modify-write: you must preserve the other bits in that register, and a blind .w= write of a guessed value can clear settings you wanted. For a home rig, pcie_aspm=off is less error-prone and does the job.

Step 5: wire the software correctly (and know which knobs are real)

This is where a lot of guides go wrong, so be precise: llama.cpp and vLLM use different cross-GPU machinery.

vLLM uses NCCL for tensor parallelism. NCCL env vars matter here:

export NCCL_P2P_DISABLE=0          # keep P2P ON (set 1 only to test the difference)
export NCCL_DEBUG=INFO             # prints the topology/transport NCCL chose — read it
# vllm serve <model> --tensor-parallel-size 2

Read the NCCL_DEBUG=INFO output on startup: it tells you whether NCCL selected a P2P transport or fell back to SHM/host. If it says host/SHM, go back to Step 2 — your ACS/IOMMU is still bouncing traffic. Note that consumer GeForce cards don’t expose NVLink, and NVIDIA’s driver historically restricts P2P over PCIe on GeForce; the community tinygrad/open-gpu-kernel-modules P2P fork exists precisely to force it on (primarily for the RTX 4090, which needs it more than the 3090). The Ampere 3090 generally does P2P over PCIe when both cards are on the same root complex — which is why Steps 1–2 matter so much.

llama.cpp does not use NCCL. It uses its own CUDA peer-access path. The relevant knobs:

llama-server -m model.gguf \
  --n-gpu-layers 99 \
  --split-mode layer \        # or "row" for tensor-parallel-style split
  --tensor-split 1,1 \        # even split across the two 24GB cards
  --flash-attn on

Watch the startup log for the line confirming it’s using both devices (using N CUDA devices / per-device layer counts). --split-mode row does a tensor-parallel-style split that’s more bandwidth-hungry across the link (great with real P2P, painful without); --split-mode layer splits by layer and is more forgiving — but as issue #20052 shows, layer split on a non-P2P topology is exactly what corrupts long-context output. If you can’t get P2P working and must run split, keep context modest until you fix the fabric.

What “fixed” actually buys you

With the fabric wired correctly, a dual RTX 3090 rig gives you two concrete wins:

• Capacity: 48GB holds a 70B model at Q4_K_M (~42.5GB of weights) that simply won’t fit on one card. On dual 3090s that lands around 7–10 tok/s on a 70B — usable for batch and overnight work, above reading speed for chat. (See our 70B-on-24GB guide for the single-card offload alternative.)
• Speed on models that fit either way: a 27B–35B model split across two properly-linked cards runs meaningfully faster than crammed onto one — the real gain is roughly 1.5–2×, not the 2× the marketing math implies, because the split adds cross-GPU traffic.

Skip the config and both wins evaporate: the second card behaves like a separate single-GPU instance, and long-context runs may corrupt. That’s the “silent killer” — nothing errors out, it just quietly underperforms.

For everyday single-model work that fits in 24GB, one 3090 is still the sweet spot (see Used RTX 3090: Still the Value King). Reach for the second card when you specifically need 48GB. And if you’d rather not manage BIOS fabric at all, renting a single 48GB card on RunPod sidesteps the whole problem for occasional big-model jobs.

FAQ

Do I need NVLink for two 3090s? No, but it helps. The 3090 supports the NVLink bridge (unlike the 4090), and real-world tests show roughly a 50% cross-GPU improvement with NVLink versus PCIe alone. If your board has the slot spacing for the bridge, it’s the cleanest way to guarantee a fast P2P path. Without it, PCIe P2P (Steps 1–4) is your route.

Why does my dual-GPU rig give garbage on long prompts but not short ones? Almost certainly a non-P2P topology (one card on a chipset x4 slot) host-staging KV-cache transfers that corrupt above a size threshold — llama.cpp issue #20052. Fix the topology or the ACS/IOMMU routing; don’t chase sampler settings.

Is iommu=off safe? On a dedicated AI box that doesn’t pass GPUs through to VMs, yes — you lose nothing and gain direct P2P. Only keep IOMMU on if you use VFIO/VM passthrough, in which case use the per-device ACS setpci route instead.

Will two 3090s make my 7B model twice as fast? No. Decode is memory-bandwidth-bound per card. Two cards give you more VRAM for bigger models, not more speed on a model that already fits on one. A 7B stays ~95 tok/s.

Does any of this apply to dual RTX 4090? The BIOS steps (IOMMU/ACS, link speed, ASPM) are identical. The 4090 additionally lacks NVLink and has stricter driver P2P restrictions, which is why the tinygrad/open-gpu-kernel-modules patch is more commonly needed on 4090 than on 3090.

Sources

1. llama.cpp Issue #20052 — Multi-GPU layer split garbage output above 2048 context on non-P2P dual RTX 3090 (ggml-org/llama.cpp)
2. NCCL Troubleshooting — IOMMU/VT-d redirects PCI peer traffic through the CPU root complex (NVIDIA, NCCL 2.30.7 docs)
3. Multi-GPU (NVIDIA) P2P capabilities and debugging tips — ACS disable via setpci ECAP_ACS+0x6.w=0000
4. tinygrad/open-gpu-kernel-modules (P2P fork) — enabling P2P over PCIe on consumer RTX GPUs
5. ASPM on Linux — kernel documentation and setpci(8) man page — Link Control register / pcie_aspm=off
6. RTX 3090 Price Tracker — July 2026 and ResalePrices RTX 3090 — used market average ~$1,254

---

Related on RunAIHome: Multi-GPU for Local AI: NVLink vs PCIe · EXO Framework: Pooling RTX 3090s · Mesh LLM + iroh: P2P inference across boxes · How to Run a 70B Model on a Single 24GB GPU. For wiring a local dual-GPU endpoint into Cline or Cursor, see aicoderscope.com; for the llama.cpp build and open-weight model picks from a FOSS angle, see aifoss.dev.

Prices and availability as of July 2026 and move weekly — verify current pricing at the retailer before buying.

Recommended Gear

• RTX 3090 — 24GB, ~$1,254 used avg (Jul 2026); two = 48GB for 70B-class models
• RTX 5090 — 32GB single-card alternative if you’d rather avoid dual-GPU fabric config