TL;DR: macOS Tahoe 26.2 added RDMA over Thunderbolt 5, which lets four Mac Studios pool 1.5TB of unified memory and run trillion-parameter MoE models like Kimi K2 at ~25 tok/s for around $40,000 total. The catch is single-stream throughput: it’s responsive for one user, not a batch-serving box. Buy it only if you truly need 1T-parameter models offline — otherwise a single high-memory Studio or a used-GPU build is far cheaper.

	4× Mac Studio cluster	Single M3 Ultra 512GB	Dual RTX 5090 build
Best for	671B–1T MoE models, offline	100B–400B MoE in one box	≤70B at top per-token speed
Total cost	~$40,000	~$9,500	~$5,000–$6,000
Unified/VRAM pool	1.5TB	512GB	64GB
The catch	Single-user throughput, TB5 wiring	Won’t fit a full 1T model	70B Q4 is the ceiling

Honest take: The RDMA update is genuinely the cheapest path to running a 1-trillion-parameter model at home in 2026 — but “cheapest” is still $40K, and 99% of home labs are better served by one 512GB Studio or a used RTX 3090.

What actually changed in macOS 26.2

For two years the story with Apple Silicon clusters was the same: the unified memory was glorious, but the moment you split a model across machines, the network killed you. Mac-to-Mac links ran over 10-gigabit Ethernet or plain TCP/IP Thunderbolt, and every token round-trip paid a latency tax of roughly 300 microseconds just moving activations between nodes. On a memory-bandwidth-bound workload, that overhead swallowed most of the benefit of adding a second machine.

macOS Tahoe 26.2 changed the physics. Apple added RDMA (Remote Direct Memory Access) over Thunderbolt 5 — the same class of technology that stitches together the GPU fabric inside the data centers running ChatGPT and Claude. RDMA lets one Mac read and write directly into another Mac’s memory without involving either machine’s CPU or kernel on the data path. AppleInsider and Jeff Geerling both documented the result: inter-node memory-access latency dropped from about 300µs to roughly 5–9µs — a 30-to-60× reduction — over a Thunderbolt 5 link that moves 80 Gb/s (versus 40 Gb/s on Thunderbolt 4 and 10 Gb/s on typical Ethernet).

That single change is what makes a four-Mac cluster behave like one big machine instead of four machines arguing over a phone line. It’s a software update — no new hardware required if you already own Thunderbolt 5 Macs.

The $40K build, exactly as tested

The build everyone is citing comes from Creative Strategies, and it’s worth being precise about it because the specs floating around online are already garbled. The trending headline says “4× Mac Studio M4 Ultra.” That’s wrong on two counts, and the corrections matter for anyone about to spend real money.

Correction 1: there is no M4 Ultra. Apple’s Ultra-tier chip in the Mac Studio is the M3 Ultra — 32 CPU cores, 80 GPU cores, and 819 GB/s of memory bandwidth. RDMA over Thunderbolt 5 works on any TB5-equipped Mac (M4 Pro Mac Mini, M4 Max Mac Studio, M4 Max MacBook Pro, and the M3 Ultra Mac Studio), but the machine that gives you 512GB in a single node is the M3 Ultra.

Correction 2: the cluster wasn’t four identical machines. The tested rig was:

Node	Chip	Unified memory	Storage
1	M3 Ultra	512GB	8TB
2	M3 Ultra	512GB	8TB
3	M3 Ultra	256GB	4TB
4	M3 Ultra	256GB	4TB

That totals 1.5TB of unified memory at a combined cost of just under $40,000. A 512GB M3 Ultra Mac Studio runs about $9,499 (the machine starts at $3,999 with 96GB; the jump to 512GB is a $5,500 upgrade), and the 256GB nodes land a few thousand lower each. Add storage upgrades and tax and you’re at the $40K figure. If you want the individual box, the Mac Studio M3 Ultra is the one to price out.

The numbers that matter: tokens per second

A trillion parameters in memory is a party trick unless the thing actually generates text at a usable speed. Here’s what the cluster produced, pulled from Creative Strategies’ write-up and Jeff Geerling’s independent benchmarks:

Model	Params (total / active)	Tokens/sec (4-node)
Kimi K2 Thinking	~1T / ~32B (MoE)	~25 tok/s
DeepSeek V3.1	671B / 37B (MoE)	~32.5 tok/s
Qwen3 235B	235B / 22B (MoE)	~32 tok/s

The pattern here is the same one that governs every memory-bound machine: what sets your speed is the active parameter count, not the total. Kimi K2 is a ~1-trillion-parameter Mixture-of-Experts model, but only ~32B parameters fire per token, which is why a 1T model runs at a speed in the same ballpark as a 235B one. All three of these are MoE architectures — dense models of this size would be far slower. We walked through why sparse activation dominates bandwidth-limited hardware in our Kimi K2 hardware guide, and it’s the whole reason this cluster is viable at all.

For context, human reading speed is roughly 7–10 tokens/sec. So 25 tok/s on a 1T model is comfortably faster than you can read — genuinely usable for interactive coding, research, and writing, one request at a time.

Why RDMA specifically doubles the multi-node numbers

The clearest before/after data comes from testing with exo, the open-source clustering tool. On the same hardware, exo with RDMA hit 26.2 tok/s across two nodes and 31.9 tok/s across four — while the non-RDMA path managed only 17.2 and 15.2 tok/s respectively. Notice the tell: without RDMA, going from two nodes to four made things slower, because network overhead grew faster than the added memory bandwidth helped. With RDMA, adding nodes finally scales the way it should. That inversion is the entire value of the 26.2 update in one data point.

Power and the NVIDIA comparison

The efficiency story is where the Mac cluster stops looking like a novelty. The full four-Studio rig draws 450–600W total under load — the Kimi K2 run came in under 500W for the whole cluster. A single NVIDIA H200 pulls about 700W by itself.

Building an equivalent 1.5TB-of-fast-memory NVIDIA rig means roughly a rack of H100s, which one analysis (implicator.ai) pegged at about $780,000 — calling the macOS update effectively a “$730,000 discount.” Rounding to round numbers: the Mac cluster costs about 5% of the NVIDIA price and draws about 3% of the power.

The honest caveat, and it’s a big one: those H100s would obliterate the Mac cluster on aggregate throughput. A datacenter rig serves dozens of concurrent users at hundreds of tokens/sec each; the Mac cluster gives you 25–32 tok/s for essentially one stream. This is a single-user or small-team appliance, not a serving cluster. If your workload is batch inference or multi-tenant serving, the comparison collapses and you should be renting — our RunPod vs. local GPU breakdown covers where cloud wins, and RunPod rents H100s by the hour if you need throughput for a burst.

How to actually build it

The software stack matured alongside the OS. Apple introduced its distributed approach at WWDC26 (session 233, “Explore distributed inference and training with MLX”). The pieces:

• MLX Distributed — Apple’s open framework for distributed ML on Apple Silicon. It supports Pipeline Parallelism over Ethernet or Thunderbolt, and Tensor Parallelism via JACCL, which specifically requires RDMA over Thunderbolt 5.
• exo 1.0 — the easiest on-ramp. It uses MLX as its inference backend and MLX Distributed for communication; once RDMA is on, exo handles node discovery and sharding for you.
• LM Studio — for a GUI front end once the cluster is serving.

Enabling RDMA is a one-time, per-machine step done in Recovery mode:

# On each Mac, boot to Recovery (hold power button ~10s → Options),
# open Terminal from the Utilities menu, then:
rdma_ctl enable
# Reboot. exo takes it from there.

Two wiring gotchas that will waste your afternoon if you miss them:

1. Every node must connect to every other node over TB5. For four Macs that’s a full mesh of cables, not a daisy chain. Make sure the cables are rated for Thunderbolt 5, not TB4.
2. On a Mac Studio, do not use the Thunderbolt 5 port next to the Ethernet jack for the cluster fabric — it doesn’t carry the RDMA link.

If four machines is more than you need, exo also clusters two Studios, or even Mac Minis, at smaller memory pools — see our exo distributed-VRAM guide for the smaller-scale setup.

When you should not build this

Most people reading this do not need a $40K cluster, and the site’s whole ethos is telling you that plainly.

If your models fit in 512GB, buy one Studio. A single M3 Ultra with 512GB runs DeepSeek V3.1 671B at heavy quantization and every MoE model up to ~400B without any of the cabling, RDMA, or multi-node complexity — and at higher per-token speed than the cluster, because there’s zero inter-node hop. Our running 100B+ models on Mac Studio guide covers this single-box path.

If you run 70B and under, a GPU build wins on speed and price. Here’s the correction to another number circulating with this story: dual RTX 5090s do not do 145 tok/s on a 70B model. Benchmarks (databasemart) put a dual RTX 5090 setup at ~27 tok/s on Llama 70B Q4 — decode is memory-bandwidth-bound, and the two cards talk over PCIe x8/x8 (NVLink died with the 3090), so you get ~1.4–1.6× scaling, not 2×. That’s still a fine 70B box for roughly $5,000–$6,000, and a single used RTX 3090 at ~$1,070 remains the value king under 24GB. We compared the Apple-vs-GPU trade-off directly in Mac Studio M3 Ultra vs dual RTX 4090.

If you need throughput or only occasionally need a giant model, rent. The 1T-parameter models this cluster targets are also available cheaply via API, and cloud GPUs cover the burst case without a five-figure capital outlay.

The build makes sense in exactly one scenario: you have a hard requirement to run trillion-parameter models offline — privacy, air-gap, or data-residency reasons — for a single user or small team, and $40K is an acceptable price for that capability. For that specific buyer, nothing else on the consumer market comes close.

FAQ

Do I need macOS 26.2 on every machine? Yes. RDMA over Thunderbolt 5 shipped in macOS Tahoe 26.2, and every node in the cluster needs it plus the one-time rdma_ctl enable in Recovery mode. Mixed OS versions won’t form an RDMA fabric.

Will this work with Mac Minis or MacBooks? RDMA works on any Thunderbolt 5 Mac — M4 Pro Mac Mini, M4 Max Mac Studio, M4 Max MacBook Pro, and M3 Ultra Mac Studio. But only the M3 Ultra Studio offers up to 512GB per node, which is why the trillion-parameter builds use it. A Mac Mini cluster is a real option for 70B–235B models at a lower memory ceiling.

Is 25 tok/s on a 1T model actually usable? For a single user, yes. It’s faster than reading speed and fine for interactive coding and chat. It is not fast enough to serve multiple concurrent users — this is a personal/small-team appliance, not a production inference server.

Why not just buy the 512GB single Studio and skip the cluster? If your models fit in 512GB, you should. One box is simpler, cheaper (~$9,500), and slightly faster per token because there’s no inter-node hop. The cluster only earns its cost when you need more than 512GB — i.e., full-size 671B–1T models with real context headroom.

Does RDMA help training too? Yes — Apple’s WWDC26 session covers distributed training as well as inference, and JACCL’s tensor parallelism benefits both. But for home labs the inference case is the realistic one; fine-tuning trillion-parameter models is still a datacenter job.

Sources

• Running a 1T-Parameter Model on a $40K Mac Studio Cluster — Creative Strategies
• AI calculations on Mac cluster get a big boost from new RDMA support on Thunderbolt 5 — AppleInsider
• 1.5 TB of VRAM on Mac Studio — RDMA over Thunderbolt 5 — Jeff Geerling
• Explore distributed inference and training with MLX — WWDC26 Session 233, Apple Developer
• exo — Run frontier AI locally (GitHub README)
• Apple just turned a software update into a $730,000 discount on AI infrastructure — implicator.ai
• Apple debuts M3 Ultra in refreshed Mac Studio with up to 512GB memory — Tom’s Hardware
• Distributed Communication — MLX Documentation
• 2×RTX 5090 Ollama Benchmark for 70B LLM Inference — Database Mart

Recommended Gear

• Mac Studio M3 Ultra — the 512GB node these clusters are built from
• RTX 5090 — the dual-card 70B alternative
• RTX 3090 — the used-market value pick under 24GB

Prices and benchmarks as of July 2026 and move quickly — verify current figures before buying. For local AI coding tools, see aicoderscope.com; for self-hosting open-source AI, see aifoss.dev.