Step 5 Preview Open Weights for Local AI: 600B Total, 27B Active, and the Four-DGX-Spark Reality Check (2026)

By RunAIHome Team · 14 min read
stepfunstep-5local-llmmoedgx-sparkhardware-guide

TL;DR: StepFun’s Step 5 Preview open weights went live on Hugging Face October 15, 2026 — a 600B-parameter MoE with just 27B active and a 1M-token context that scores 44 on the Artificial Analysis Intelligence Index, tying Kimi K3 at roughly a fifth of the weights. The catch: the smallest proven local setup is a 341GB quant spread across four DGX Sparks, delivering 25-30 tok/s.

4× DGX Spark clusterMac Studio M5 Ultra 256GBStepFun API / OpenRouter
Best forThe only measured local run so farQ2 on one box — on paper, untestedActually using Step 5 today
Price / Cost~$18,800 (4 × $4,699)$9,499$1.00/M in, $2.70/M out
Speed25-30 tok/s single stream (measured)~25-45 tok/s (estimated ceiling math, no benchmark yet)~85 tok/s
The catch12-minute load, cluster networking, EXL3-only quant~217GB Q2 GGUF isn’t publicly downloadable yetIt’s not local

Honest take: Step 5 Preview is the most interesting architecture of October’s open-weight wave — frontier-index intelligence with the lightest per-token memory traffic of any frontier MoE. It is also, today, the least runnable: no official GGUF, no Ollama path, no Mac benchmark. Use the API at $1/$2.70, keep your 24GB card on Nemotron 3.5 Bolt, and let the quant ecosystem catch up before spending a dollar on hardware for this model.

The weights actually shipped this time

StepFun announced Step 5 Preview on September 20, 2026 as an API-only model and promised open weights for October 15. In a month where “announced” and “released” have been drifting apart — Mistral Large 4’s weights are still pending until October 31, and a leaked 1.2TB copy of Step 5 itself sat in an unofficial Hugging Face account for weeks — StepFun hit its date, give or take a confusing first day. Trackers reported the official stepfun-ai/Step-5-Preview-BF16 repo still empty in the early hours of October 15; by StepFun’s announcement later that day, BF16 and FP8 checkpoints were live on Hugging Face and ModelScope under the official stepfun-ai namespace, linked from the company’s own GitHub repo.

That lands Step 5 Preview in the same 24-hour window as Meta’s Muse Spark Open 816B and NVIDIA’s Nemotron 3.5 Bolt — three very different answers to what “open” means for home labs. Bolt shipped a 22GB quant that fits a 24GB card. Muse Spark shipped official GGUFs that need 256GB+. Step 5 shipped raw BF16/FP8 checkpoints and a datacenter deployment guide, and left everything below 8× H200 to the community.

What 600B total / 27B active buys you

Step 5 Preview is a sparse Mixture-of-Experts model: 600B total parameters, 27B active per token, up to 1M tokens of context (input and output combined), text and image input, text output. Recommended sampling is temperature=1.0, top_p=0.95, top_k=0 — the same temperature-1.0 convention as other recent reasoning MoEs; lower it and you’re off the tested path.

The 27B-active figure is the headline for anyone who thinks in memory-bandwidth terms. Decode speed on MoE models is bounded by how many bytes each token actually reads, not by total parameter count. Among the frontier open MoEs released this fall:

ModelTotal paramsActive paramsActive fraction
Step 5 Preview600B27B4.5%
Meta Muse Spark Open816B42B5.1%
Mistral Large 41.05T49B4.7%
Kimi K32.8Thigher still—

Lightest per-token traffic of the group — at a given quantization, Step 5 decodes faster on the same hardware than any of them. The intelligence-per-byte-read story holds up too: Artificial Analysis scores Step 5 Preview 44 on its Intelligence Index, tying Kimi K3 (44), one point under GLM-5.3 (45), and well clear of Muse Spark Open (39). It briefly climbed to No. 2 on OpenRouter’s weekly usage chart after the September API launch. Two honest caveats ride along: AA flags the model as unusually verbose (it burned ~160M tokens completing the index evaluation, $918 of API spend), and the strongest benchmark numbers — DeepSWE, GDPval, the finance suite — are vendor-published with no independent replication yet.

None of that changes the capacity problem: all 600B parameters must sit in memory somewhere, active or not.

The memory wall, checkpoint by checkpoint

FormatSizeWhat officially runs itStatus
BF16~1.2TB8× B300 or 16× H200 (two nodes)Official, live Oct 15
FP8~600GB (computed: half of BF16)4× B300 or 8× H200Official, live Oct 15
EXL3 hybrid (K4 experts / K8 body)341.4GB4× DGX Spark, vLLM TP4Community, measured
GGUF Q2_K202.45 GiB (~217GB), 2.9 bpw256GB-class unified memory, in theoryCommunity repo, not a general release
GGUF IQ1_S111.32 GiB (~120GB), 1.6 bpwOne 128GB GB10-class box, per the repoSame repo, same caveat

Read the bottom two rows carefully before planning a build around them. As of October 17 there is no official GGUF, no Unsloth or bartowski release, and no Ollama path. The only GGUF conversion with published sizes is a single community repo (vcruz305) that describes itself as private and “not a general release,” and it quantizes the text trunk only: llama.cpp’s step35 architecture — merged for Step 3.5-Flash back in February 2026 (PR #19283) and extended with multi-token-prediction support in June (PR #24340) — has no runtime for Step 5’s vision encoder or MTP draft heads, so those tensors ship as dead weight in companion files. A 1.6-bpw quant of the whole model fitting a 128GB box is a genuinely interesting claim. It is also, today, a claim you cannot download and verify.

If you want to understand why a 2-bit, 217GB file is the realistic floor rather than some 24GB miracle quant, the math is the same as every big MoE we’ve covered: 600B parameters at ~2.9 bits per weight is ~217GB before KV cache, and SSD-streaming engines that fake extra capacity collapse to single-digit tok/s when the expert working set outruns RAM. Strata’s 2-bit 125B trick doesn’t scale to 600B on a gaming GPU either.

The one real local run: four DGX Sparks, 25-30 tok/s

The most useful Step 5 data point anywhere right now isn’t StepFun’s — it’s a merged pull request in a public homelab registry (sybil-solutions/local-ai-registry #172, measured October 4-6 on the leaked checkpoint, nine days before the official release). The builder (0xSero) ran Step 5 Preview across four NVIDIA DGX Sparks — 512GB of pooled unified memory — using a custom vLLM build, a 341.4GB EXL3 hybrid quant (K4 routed experts, K8 decode body, BF16 prefill body), tensor parallelism over the Sparks’ ConnectX-7 RoCE fabric, and 2-token MTP speculative decoding.

Measured results:

  • Decode, single stream: 21.3 tok/s in the registry’s controlled speed gate; 25-30 tok/s in general use (code and prose).
  • Decode, 4 concurrent streams: 66.5 tok/s aggregate on code (~19 tok/s per stream).
  • Prefill: 1,365-1,565 tok/s warm on an 8K prompt; 888 tok/s on a ~222K-token prompt.
  • Context: 262,144 tokens served and verified (a 261,632-token passphrase-recall prompt passed 4/4; one token over the limit returns HTTP 400). The 1M-token window survives on paper only — KV cache for 262K already reserves 20GB per Spark.
  • Load time: 12-13 minutes with a warm page cache.

Sanity-check against the bandwidth ceiling, the way we check every number on this site: 341.4GB × (27B active ÷ 600B total) ≈ 15.4GB read per token. Four Sparks at 273 GB/s each give ~1,092 GB/s aggregate, so the hard ceiling is ~71 tok/s single-stream — the measured 25-30 is 35-42% of ceiling, exactly the efficiency band real multi-node MoE deployments land in. The numbers are plausible, and they also tell you the ceiling: this cluster will never do 70 tok/s on this model.

The run surfaced a real bug worth knowing if you attempt anything similar. With vLLM’s shared-expert stream enabled, the EXL3 MoE kernel and the shared-expert GEMMs deadlocked an entire rank — the cluster hangs mid-generation, no error, no crash:

# one rank stops responding mid-decode; nvidia-smi shows 100% util, no progress
# fix: serialize shared-expert work onto the main CUDA stream
export VLLM_DISABLE_SHARED_EXPERTS_STREAM=1

Total hardware for this path: 4 × $4,699 = ~$18,800 of NVIDIA DGX Spark, plus the ConnectX cabling, for 25-30 tok/s. For calibration: $9,499 of Mac Studio M5 Ultra runs Muse Spark Open’s 2-bit quant at ~18 tok/s on one box, and a used RTX 3090 at $1,399-1,450 runs Nemotron Bolt — a 74B agent model NVIDIA ships at 2-bit — at an estimated 70-80 tok/s. Step 5 is the smartest model of the three and the worst dollars-per-token-per-second of the three. That’s the trade.

The Mac path exists on paper and nobody has walked it

A 256GB M5 Ultra Mac Studio ($9,499, 1.2TB/s) can hold the ~217GB Q2_K in principle, with the usual iogpu.wired_limit_mb ceiling-raising and almost no room for KV cache. Ceiling math says 1,200 GB/s ÷ ~9.8GB-per-token ≈ 123 tok/s theoretical, so 25-45 tok/s at the 20-35% efficiency Apple Silicon typically achieves on big MoEs — our estimate, no measurement exists. Nobody has published a Step 5 Mac run because the Q2_K isn’t publicly downloadable and MLX has no Step 5 port. The 512GB M5 Ultra, which would hold a 4-bit quant the way it holds Muse Spark, is still not orderable — Apple said late October, and as of October 17 the configurator still stops at 256GB with no price announced.

If the Mac route is your plan, the honest move is the same one we gave for Muse Spark: wait for the 512GB tier’s price and for a real GGUF/MLX release, then decide. Buying a $9,499 machine today for a quant you can’t download is speculation, not a build.

The license is fine for you, with one trap for fine-tuners

The weights ship under the StepFun Model License 1.0 — not Apache 2.0, but closer to it than Meta’s Muse license. Reading the actual license file:

  • Home lab and internal commercial use: unconditionally fine. Perpetual, worldwide, royalty-free rights to use, modify, quantize, and redistribute, with a patent grant. Section 2(c) explicitly exempts “internal use that does not make the Model… available to any third party.”
  • The “Powered by StepFun AI” attribution only triggers if your product exceeds 100M monthly active users or $20M revenue in a calendar month.
  • A separate commercial license is required for hardware makers shipping the model as a device feature, and for companies running a model-as-a-service business with >$20M revenue over twelve months. Selling API access to Step 5 at scale needs StepFun’s paper; running it for your own team doesn’t.
  • The trap: output training. Any model “trained to a material extent on outputs of the Model” is deemed a Derivative Work and inherits this license. If you’re distilling Step 5 traces into your own small model, your model is no longer cleanly yours. Apache-licensed alternatives don’t carry that clause.

What this release actually changes

For anyone with a budget under $10K: nothing, today. The model you can’t load doesn’t compete with the model you can. What matters is the direction — three of October’s four open releases (Bolt 74B/9B, Muse Spark 816B/42B, Step 5 600B/27B) push the active-parameter fraction down toward 4-5%, which means every GB/s of bandwidth you own keeps buying more intelligence. A 27B-active frontier model is also the best possible news for the 128GB unified-memory class — if a reputable 1.6-2.0 bpw GGUF materializes and holds up on quality, a GMKtec EVO-X2 128GB ($3,499-3,649) or a DGX Spark on its own becomes a single-box Step 5 machine at ~10-17 tok/s (ceiling math again: ~120GB × 4.5% ≈ 5.4GB/token against 256-273 GB/s). That’s a real possibility within weeks, and it’s worth waiting for rather than pre-buying.

What to actually buy

Prices as of October 2026, all verified in this article:

Your situationThe movePriceWhere
You want Step 5’s output todayStepFun API via OpenRouter$1.00/M in, $2.70/M outOpenRouter
You own a 24GB cardRun Nemotron 3.5 Bolt instead, revisit Step 5 when GGUFs mature$0Our Bolt guide
You want the biggest MoE one buyable box runs nowGMKtec EVO-X2 128GB (120B-class today, Step 5 IQ1 candidate if real GGUFs land)~$3,499-3,649Check price
You’re committed to Step 5 locally, now4× NVIDIA DGX Spark + vLLM EXL3 (25-30 tok/s, measured)~$18,800Check price
You want to test big-MoE workloads before any purchaseRented multi-GPU instancepay per hourVast.ai

FAQ

Can a single 24GB GPU run Step 5 Preview? No, and no quantization will change that. Even at 1.6 bits per weight the model is ~120GB. A 24GB card holds a fifth of it; offloading the rest to system RAM or SSD drops decode to unusable single digits. On 24GB, Nemotron 3.5 Bolt and Qwen3.6-35B-A3B remain the ceiling.

Is there an Ollama or LM Studio path? Not as of October 17, 2026. llama.cpp’s step35 architecture loads the text trunk of community conversions, but there is no official GGUF, no major-quantizer release, and no vision or MTP support in llama.cpp. If that changes — Unsloth dynamic quants would be the signal — the 128GB-box math in this article is the one to revisit.

Is the license safe for a home lab or small business? Yes. Use, modification, quantization, and internal commercial deployment are unconditional. The restrictions bind device manufacturers, $20M+ MaaS operators, and 100M-MAU products. The one clause that reaches small players: models distilled from Step 5 outputs inherit the license.

How does it compare to Kimi K3 and GLM-5.3? On Artificial Analysis’s index it ties K3 (44 vs 44) and trails GLM-5.3 by one point (45) — with 600B parameters against K3’s 2.8T. Per byte of memory you need, Step 5 is the most intelligence currently available in open weights. GLM-5.3 and K3 still win on independent agentic benchmarks, and Step 5’s own agentic scores are vendor-reported so far.

Should I wait for the 512GB M5 Ultra to run it on a Mac? Wait for two things, not one: Apple’s actual price on the 512GB tier (late October, still unannounced), and a downloadable GGUF or MLX port with a measured tok/s. Until both exist, the Mac path is arithmetic, not an option.

Products linked in this article:

  • NVIDIA DGX Spark — $4,699 each; four of them are the only measured local Step 5 setup
  • GMKtec EVO-X2 128GB — $3,499-3,649; the single-box candidate if sub-2-bit GGUFs become real
  • Used RTX 3090 24GB — $1,399-1,450; still the right buy for everything that isn’t a 600B model

For local coding setups that pair a big-model API with a local autocomplete model, see our sister site’s Continue.dev + Ollama stack guide, and aifoss.dev’s Ollama review for the serving layer.

Sources

Last updated October 17, 2026. Prices and specs change; verify current rates before purchasing. Estimated speeds are bandwidth-ceiling calculations, not benchmarks, and are labeled as such.

Was this article helpful?

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.