TL;DR: Inkling is Thinking Machines Lab’s first open model — a 975B-total / 41B-active multimodal MoE under Apache 2.0, and the top-scoring U.S. open-weights model on the Artificial Analysis Intelligence Index. But the smallest 1-bit quant needs ~290GB of memory and a real 4-bit build is ~550GB, so no single consumer GPU touches it. Your three honest options are the API, a rented multi-GPU pod, or a 512GB Mac Studio running a degraded 1-bit quant.

	Tinker / cloud API	RunPod multi-GPU	Local big-memory build
Best for	99% of home labs	Bursty heavy use, tuning	Data-can’t-leave, always-on
Cost	~$1.87 / $4.68 per M tok (launch rate)	~$2.89/hr per H100, 8× for Q4	~$9.5k (512GB Mac) to ~$40k (cluster)
Speed	Managed, fast	20–40 tok/s	~5–15 tok/s (1–2 bit / CPU offload)
Memory needed	None	~550GB VRAM (rented)	290–550GB unified/RAM
The catch	Prompts leave your machine	Billed by the hour	Slow, degraded quant, expensive

Honest take: For nearly every home-lab reader, the right move today is the API or a rented RunPod pod — this is a data-center model that happens to have open weights, not a card you buy. The model actually worth waiting for is Inkling-Small (276B-A12B): when its weights drop, 12B active parameters make it fast, and a 2-bit quant could fit a single high-memory machine.

What Inkling actually is

Inkling is the first in-house model from Thinking Machines Lab, the research company founded by former OpenAI CTO Mira Murati. It shipped on July 15, 2026 under a full Apache 2.0 license, with weights on Hugging Face and same-day fine-tuning through the company’s Tinker platform.

The architecture is a sparse Mixture-of-Experts:

• 975 billion total parameters, ~41 billion active per token (975B-A41B)
• A 66-layer decoder-only transformer
• 256 routed experts + 2 shared experts per layer, with 6 routed experts active on each forward pass
• Native multimodality across text, images, audio, and video, trained on ~45 trillion tokens
• A 1M-token context window on the open weights (256K through the Tinker API)
• Controllable “thinking effort” — you dial reasoning depth up for hard problems or down for speed

Thinking Machines is explicit that Inkling isn’t meant to be the single strongest model on every axis. It’s positioned as a balanced, multimodal, customizable base — something organizations fine-tune for their own workloads rather than a finished chat product. That framing matters for home labs, because it means the interesting local story is fine-tuning and distillation, not just inference.

The benchmarks: leading U.S. open weights, still chasing China

Inkling debuted at 41 on the Artificial Analysis Intelligence Index, making it the strongest U.S. open-weights model on that composite as of mid-July 2026. The detail underneath is more interesting than the headline:

• AIME 2026: 97.1% — edging DeepSeek V4 Pro (96.7%) on competition math
• SWE-bench Verified: 77.6% — behind DeepSeek V4 Pro (80.6%) on agentic coding
• GPQA Diamond: 87.9% — behind Kimi K2.6 (91.1%)
• BrowseComp: 77.1% vs Kimi K2.6’s 83.2%
• HLE with tools: 46.0% vs Kimi K2.6’s 54.0%
• SimpleQA Verified: 43.9% — well behind DeepSeek V4 Pro (57.0%) on factuality
• IFBench: 79.8% — ahead of Kimi K2.6 (76.0%) on instruction following

So it’s the best American open model, but the strong Chinese MoEs (Kimi K2.6, DeepSeek V4 Pro) still win several coding, reasoning, and factuality tests. For a picture of which open models actually fit a single consumer GPU, our open-source LLM shootout is the companion read — Inkling, like the other trillion-class MoEs, is not on that list.

The one number that quietly matters most for cost is token efficiency: Inkling averaged ~25,000 output tokens per Intelligence Index task, versus ~37,000 for DeepSeek V4 Pro and ~38,000–43,000 for Kimi K2.6 and GLM-5.2. At the same per-token price, a model that thinks in 25k tokens instead of 40k is roughly a third cheaper per task — that efficiency is a real part of the value, not a footnote.

The VRAM reality: it’s a storage problem, not a speed problem

MoE routing means only ~41B parameters are computed per token, but all 975B weights still have to be resident in memory — the router picks a different set of experts on every token, so the whole model must be loaded. That’s the wall, and it’s the same wall we hit with the Qwen3-Coder 480B guide: active-parameter count sets your speed, total-parameter count sets your memory bill.

Here are the community GGUF sizes from Unsloth’s unsloth/inkling-GGUF dynamic quants. The 1-bit and 8-bit figures are Unsloth’s stated memory requirements; the 2-bit and 4-bit rows are estimates from the parameter count and typical bits-per-weight, so treat them as planning numbers until you check the repo:

Quant (Unsloth)	Approx. size	Quality signal	Where it fits
UD-IQ1_S (~1-bit)	~290GB	~74.2% of full accuracy, lossy	512GB Mac Studio / 384GB+ RAM build
UD-Q2_K_XL (~2-bit)	~350GB (est.)	~81% of full accuracy	512GB Mac Studio (tight) / big-RAM build
Q4_K_M (~4-bit)	~550GB (est.)	Near-full quality	Mac cluster / 8× H100 80GB
Q8_0 (8-bit)	~975GB+	Full precision	900GB+ RAM server / datacenter

Unsloth’s own note is that even the aggressive dynamic quants hold up — an 8-bit + 6-bit mixture lands within RMSE of 1e-4 on dequantization — but a 1-bit quant at ~74% accuracy is a different model than the one on the leaderboard. Don’t quote the 41 Intelligence Index score and then run IQ1_S; you’re not running the same thing.

The four ways to actually run the 975B

1. A single 512GB Mac Studio (~$9,499). The Mac Studio M3 Ultra with 512GB unified memory and 819 GB/s bandwidth is the only single purchasable box that holds Inkling at all — and only at 1-bit (~290GB) with headroom, or 2-bit (~350GB) tight. Expect low-double-digit tokens/sec on a 41B-active MoE at that bandwidth, and remember you’re running the degraded quant.

2. A Mac Studio cluster (~$40k). Four M3 Ultra Studios pooled to ~1.5TB via RDMA over Thunderbolt 5 will hold the ~550GB Q4 quant comfortably. This is the cheapest path to a good quant of a 1T-class model at home, and we walked the full build in the Mac Studio cluster guide — including the honest caveat that it’s still single-user throughput for $40k.

3. A big-RAM CPU-offload server. A 512GB DDR5 workstation running llama.cpp with expert-offload flags will load IQ1_S or Q2, at single-digit-to-low-teens tokens/sec. Cheaper than the Mac cluster, slower, and you’re back on 1–2 bit quality.

4. Rent it on RunPod. Eight H100 80GB cards give you 640GB of VRAM — enough for the ~550GB Q4 with room for KV cache — at roughly $2.89/hr per card. For evaluating Inkling, fine-tuning a checkpoint through Tinker, or a bursty weekend project, renting is far cheaper than a $40k cluster you’d use two hours a day. This is the same rent-vs-buy math we lay out in RunPod vs local GPU.

What about used 3090s? At ~550GB for Q4 you’d need roughly two dozen used RTX 3090 cards — over $30,000 at the July 2026 market average of ~$1,254 each (fair range $1,194–$1,306 across 319 eBay listings) — before you even solve the PCIe-lane, power, and cooling nightmare of a 24-GPU rig. It’s the wrong tool. A single used 3090 is a fantastic value for models that fit 24GB (see why it’s still the value king); it is not a brick in a 975B wall.

Inkling-Small (276B-A12B) is the model home labs should actually watch

Buried under the 975B headline is the variant that matters for consumer hardware: Inkling-Small, a 276B-total / 12B-active MoE. Thinking Machines says it matches or exceeds the big model on many benchmarks thanks to a better pre-training recipe, and it’s aimed squarely at cost- and latency-sensitive work — coding, LLM-based grading, and synthetic-data generation.

Two things make it the interesting one:

• 12B active parameters means decode reads only ~12B weights per token, so it’s genuinely fast on bandwidth-limited hardware — the same reason a 30B-A3B MoE outruns a dense 32B on the same card.
• 276B total is roughly 3.5× smaller than the 975B. A 4-bit quant lands near ~155GB and a 2-bit quant near ~85GB (parameter-math estimates) — meaning a 2-bit Inkling-Small could plausibly fit a single 96GB or 128GB machine, and a 4-bit would fit a 256GB Mac Studio or a mid-range multi-GPU box.

The catch, as of July 16, 2026: Inkling-Small is still a preview and its full weights have not been released. There’s no public checkpoint, no firm date, and no confirmed GGUF yet. So the practical advice is: run the 975B via API today if you need it, and set a reminder to revisit when Inkling-Small weights land — that’s the release that could actually live on your hardware.

Cost math: API vs owning the metal

Through Tinker, Inkling launched at roughly $1.87 per million prefill tokens and $4.68 per million sample tokens at 64K context (a limited-time 50% launch discount, with rates set to rise around July 17, 2026). Combined with its ~25k-tokens-per-task efficiency, that makes a lot of real workloads cheap.

Say you run 10 million tokens a month of genuine Inkling work. At the launch sample rate that’s on the order of $40–$50/month — and you carry zero hardware, power, or maintenance. To beat that locally you’d amortize a $9,500 Mac Studio (running the degraded 1-bit quant) over years, or a $40k cluster that only makes sense if you’re saturating it daily. The buy case isn’t cost; it’s privacy and offline control — code or data that genuinely cannot leave your building. If that’s you, rent a pod first to confirm the model earns its keep before you spend on hardware.

For wiring a local or rented Inkling endpoint into a coding workflow, the setup patterns on aicoderscope.com (Cline, Cursor, and Continue against a self-hosted backend) carry straight over, and aifoss.dev covers the Apache 2.0 self-hosting angle in more depth.

FAQ

Can I run Inkling on an RTX 4090 or 5090? No. A single 24GB or 32GB card can’t hold even the 1-bit quant (~290GB). You’d need a 512GB Mac Studio, a multi-GPU cluster, or a big-RAM server — or just use the API.

Is Inkling really open source? The weights are released under Apache 2.0, which permits commercial use and modification. That’s a genuinely permissive license — more so than the OpenMDW or community licenses attached to several recent “open” model drops.

How does Inkling compare to Kimi K2.6 or DeepSeek V4 Pro? Inkling is the top U.S. open-weights model (41 on the Artificial Analysis Intelligence Index) and wins on competition math (97.1% AIME 2026) and instruction following. The strong Chinese MoEs still edge it on agentic coding (SWE-bench), factuality (SimpleQA), and hard reasoning (GPQA Diamond, HLE).

What’s the difference between Inkling and Inkling-Small? Inkling is 975B-total/41B-active; Inkling-Small is 276B-total/12B-active. Small is faster and much smaller, reportedly matching the big model on many benchmarks — but its weights are still in preview and not yet downloadable.

Why does a 975B model need 290GB+ even though only 41B parameters are “active”? Active parameters set compute per token; total parameters set memory. MoE routing selects different experts every token, so all 975B weights must stay resident in memory even though only ~41B are used per step.

Should I buy hardware for Inkling now? Only if you have a hard privacy/offline requirement and daily heavy use. Otherwise use the API or rent a RunPod pod, and wait for Inkling-Small weights before considering a local build.

Sources

1. Inkling: Our open-weights model — Thinking Machines Lab
2. Inkling Model Card — Thinking Machines Lab
3. Thinking Machines has released Inkling, the new leading U.S. open weights model — Artificial Analysis
4. Thinking Machines Lab Releases Inkling: 975B-Parameter Open-Weights Multimodal MoE — MarkTechPost
5. Inkling — How to Run Locally — Unsloth Documentation
6. unsloth/inkling-GGUF — Hugging Face
7. Thinking Machines amps up its bet against one-size-fits-all AI with Inkling — TechCrunch
8. RTX 3090 24GB Price: Updated July 2026 — GPUDojo

Recommended Gear

• Mac Studio M3 Ultra — the 512GB unified-memory config (~$9,499, 819 GB/s) is the only single purchasable box that holds Inkling at all, and the building block for a Thunderbolt 5 cluster.
• RTX 3090 — not for Inkling (you’d need ~two dozen), but ~$1,254 used and still the value king for any model that fits 24GB.

Prices and benchmarks as of July 2026 and will move. Verify current figures before buying.