TL;DR: Janas-LLM is a GPL-3.0 C engine that runs the 48.4 GB Qwen3-Next-80B-A3B at 24 tok/s (37 tok/s with drafts) on a 32 GB laptop by streaming MoE experts from the NVMe SSD — the same model llama.cpp writes at 2.8 tok/s on identical hardware. The catch: one developer, one tested machine, and the README calls it “an experiment, not a product.”

	Janas-LLM on your laptop	Used RTX 3090 24GB	Strata on a 12GB GPU
Best for	80B-class quality on hardware you already own	Speed + the mainstream CUDA/Ollama stack	125B-class models, 64GB RAM rigs
Price / Cost	$0 (software) + a fast NVMe if you lack one (~$370–$390 for a 990 Pro 2TB)	$1,399–$1,450 (October 2026 street)	$0 (MIT) + 64GB RAM
The catch	Verified on exactly one laptop; experiment-grade stability	48.4 GB model doesn’t fit 24GB — you’re running 30–35B models instead	Locked to one model family, aggressive 2-bit quant

Honest take: Install it tonight if you have 32 GB of RAM and an NVMe drive — it’s free and the numbers are real on at least one machine. But don’t cancel a GPU purchase over a project with one tested laptop and a changes-without-notice CLI.

A solo-developer C engine called Janas-LLM front-paged Hacker News on October 13, 2026 with a claim that reads like a typo: Qwen3-Next-80B — a 48.4 GB Q4_K_M file — writing 24 tokens per second on a laptop with 32 GB of RAM, against 2.8 tok/s for llama.cpp on the same machine. With the model’s built-in draft heads enabled, chat replies hit 37 tok/s.

We pulled the repo, the wiki benchmarks, and the hardware notes to check what’s actually being measured, where the speed comes from, and who should care. The claim holds up better than most HN-headline numbers — with caveats that matter more than usual.

What Janas-LLM actually is

Janas-LLM is the first component of “janas,” written in C with no dependencies beyond the C library, licensed GPL-3.0-or-later, by developer Maurizio Cammalleri. As of October 14, 2026 the repo sits at 354 stars and 22 forks. It ships prebuilt binaries for Linux x86-64 (Ubuntu 22.04 / Debian 12+) and Windows 10 (1903+) / 11. macOS is not supported.

It runs a fixed catalog of 14 converted models — Qwen3-Next-80B-A3B, Qwen3-Coder-Next, Qwen3.6-35B-A3B, Qwen3-30B-A3B, Gemma 4 26B-A4B, Qwen3.8-27B, and smaller — published pre-converted on Hugging Face under the same account. You don’t point it at an arbitrary GGUF; if your model isn’t in the catalog, this engine isn’t for you yet.

The pitch, verbatim from the README: run “LLMs bigger than your RAM.” The weights stay on disk. The experts each token needs are read from the NVMe SSD with O_DIRECT, and the experts that keep coming back stay in an in-memory cache that takes whatever RAM is free when the model opens.

Getting started is a two-command affair:

$ wget https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz
$ tar xzf janas-linux-x86_64.tar.gz && cd janas
$ ./janas-chat

GPU acceleration is optional, via Vulkan (sudo apt install libvulkan-dev glslang-tools vulkan-tools before building from source). JANAS_GPU=0 forces CPU-only.

Why an 80B model runs at all: the MoE math

This works for one architectural reason, and it only works for MoE models. Qwen3-Next-80B-A3B holds 80 billion total parameters but activates only 3 billion per token. At Q4_K_M, that’s roughly 1.8 GB of weights read per token — not 48.4 GB. A dense 80B model would genuinely need all 48 GB per token, and no SSD on earth saves you.

Even 1.8 GB per token would cap a 5.7 GB/s NVMe drive at about 3 tok/s if every expert came off the disk cold. The reason Janas-LLM reports 24 tok/s is the expert cache: on the benchmark run, roughly 19 GB of the most-reused experts sat in RAM, so most per-token reads were served at memory speed and the SSD only filled the misses. The engine reads the disk at full speed with O_DIRECT, skipping the OS page cache.

That dependency on free RAM is also the first honest limit. The wiki notes that with a browser and an editor open, the same model kept about three quarters of its speed and waited five times longer on the disk. The cache takes what’s free at model-open time; --cache <GiB> caps it if the engine is pushing your desktop into swap — which the README admits it can do.

If you want to sanity-check what fits your own machine, our VRAM calculator does the capacity math for conventional (non-streamed) loading.

The benchmarks, decoded

All figures below are from the project wiki, measured October 9–13, 2026 on the developer’s single test machine: Debian 13, Intel Core Ultra 9 185H, integrated Arc GPU, 32 GB DDR5, 1 TB NVMe at ~5.7 GB/s reads. The llama.cpp side is build 2b18470 (September 18, 2026), Vulkan backend, llama-bench -p 1024 -n 16 -ngl 99 -t 12. “Reading” is prompt processing on 1,024 tokens; “writing” is decode, no draft models on either side.

Model (Q4_K_M)	File size	Read: Janas	Read: llama.cpp	Write: Janas	Write: llama.cpp
Qwen3-Next-80B-A3B	48.4 GB	62 tok/s	16 tok/s	24 tok/s	2.8 tok/s
Qwen3-Coder-Next	48.4 GB	66	14	22	2.2
Qwen3.6-35B-A3B	22.3 GB	104	79	19	12
Qwen3-30B-A3B	18.6 GB	135	122	32	15
Gemma 4 26B-A4B	18.3 GB	84	76	18	12
Qwen3.8-27B (dense)	17.6 GB	30.6	25.6	4.4	2.1
Qwen3.5-9B	5.7 GB	111	78	14	8.7

Three things jump out of that table once you stop staring at the 80B row.

The 8.6× win exists only where the model doesn’t fit RAM. On the two 48.4 GB models, llama.cpp is doing desperate mmap paging on a 32 GB machine — the wiki notes its usual build ran out of memory with the GPU enabled, and the 2.8 tok/s figure is CPU-only (2.0 with GPU). Janas-LLM’s expert streaming is purpose-built for exactly this overflow case, and that’s where it earns the headline.

Where the model fits in RAM, the gap shrinks to 1.3–2×. Qwen3-30B-A3B: 32 vs 15 tok/s. Respectable engineering, not a revolution — and llama.cpp runs thousands of models while Janas runs 14.

Dense models barely benefit. Qwen3.8-27B writes at 4.4 tok/s vs 2.1 — both unusable. Expert streaming has nothing to stream when every token touches every weight. If your target model is dense, stop reading and go look at a used RTX 3090.

And the headline 37 tok/s? That’s a separate measurement: janas-bench on a 64-token chat reply with the model’s own multi-token-prediction draft heads enabled, at 74–89% draft acceptance on the Qwen models. It’s a legitimate chat-speed number — the same trick Ollama and llama.cpp use for supported models — but the apples-to-apples no-draft decode figure is 24 tok/s. Both are faster than you can read.

For calibration against site canon: a used RTX 3090 runs Qwen3.6-35B-A3B at 107 tok/s with Ollama — 5.6× Janas-LLM’s 19 tok/s on the same model. SSD streaming buys you capacity you didn’t pay for, not speed.

The problem you will actually hit (and the fixes)

The repo is refreshingly blunt about failure modes, and two came up in testing within the first week of public attention.

Memory pressure first. The engine grabs free RAM for its expert cache at model-open, and the README warns it “can fill memory and push the desktop into swap.” If your session turns to molasses the moment the model loads:

$ ./janas-chat --cache 12

caps the expert cache at 12 GiB and leaves the rest of your RAM alone — at some cost in tok/s, since more expert reads fall through to the SSD.

GPU driver resets second. One early GPU run “took the desktop with it,” in the project’s own words, and a tester’s Quadro RTX 5000 produced corrupted output (since fixed in code, but verified only on the code path, not on that card — the developer doesn’t own one). Tested GPUs so far: the dev laptop’s integrated Arc, an RTX 5080 (+13% prompt, +11–16% replies, per issue #15), and a Radeon 780M (prompt 477 vs 92 tok/s CPU-only, per issue #16). Every other card is officially “unknown.” The sane sequence on untested hardware:

$ ./vkcheck        # zero-copy, int8 dot product, bit-identical float tests
$ ./janas-bench    # short, bounded run
$ JANAS_GPU=0 ./janas-chat   # fallback if either misbehaves

Run the bounded benchmark before a long chat session, and keep JANAS_GPU=0 in your pocket.

Who this is for — and who it isn’t

The winner profile is specific: you have a 2024-or-later laptop or desktop with 32 GB of RAM and a decent NVMe drive, and you want to taste 80B-class output quality without spending a dollar. That machine was, until last week, a 9–14B-model machine on the standard Ollama path. Janas-LLM moves it two model classes up at usable chat speed. Developers already running local coding stacks get the clearest payoff: Qwen3-Coder-Next at 22 tok/s on a work laptop is a real agentic-coding backend, not a demo.

It is not for 16 GB machines — the developer flags that as an untested gap, with quality-for-memory thresholds measured only at 32 GB. It is not for Mac users (unsupported; MoltenVK appears in the docs as an aspiration). It is not for SATA-SSD desktops — “should work, simply wait more,” verified by nobody. And it is not a replacement for a serving stack: no OpenAI-compatible API maturity, a CLI that “changes without notice,” and stability the README itself labels “not verified.” If you need boring reliability, Ollama remains the default for good reasons.

It’s also worth placing in the 2026 SSD-streaming landscape, because this is now a genre. AirLLM streams dense-model layers through a 4 GB GPU at ~20 seconds per token — a party trick. Strata streams experts for one 125B model family into a 12 GB GPU, but wants 64 GB of RAM and its headline numbers lean on a 2-bit quant. Janas-LLM is the first of the three that needs no discrete GPU at all and runs a Q4 quant at chat speed on a stock 32 GB machine. The pattern across all three: the NVMe drive is now a compute-relevant component, something we flagged in our SSD guide back when it only affected load times.

What to actually buy

Prices as of October 2026, all verified this run:

Your situation	The move	Price	Where
32GB RAM + NVMe already in the machine	Janas-LLM, tonight — nothing to buy	$0	GitHub
32GB RAM but a slow or full SSD	Samsung 990 Pro 2TB (~7.4 GB/s reads)	~$370–$390	Check price
Want 80B-class quality AND speed, mainstream stack	Used RTX 3090 24GB (936 GB/s, 107 tok/s on 35B-A3B)	$1,399–$1,450	Check price
Undecided — test big-MoE workloads before spending	Rented RTX 3090	from $0.07/hr	Vast.ai

One note on the SSD row: the 990 Pro spent early 2026 at $640 during the flash price spike, so $370–$390 is the normal-ish price returning, not a fire sale. Don’t buy a drive only for Janas-LLM — buy it if your drive is the bottleneck for model storage generally.

FAQ

Is the 37 tok/s number real? It’s real but conditional: janas-bench on October 10, 2026, Qwen3-Next-80B-A3B with the model’s draft heads on, 64-token chat reply, mean of rounds on one machine. The no-draft decode figure is 24 tok/s. Both beat llama.cpp’s 2.8 tok/s on the same laptop by a wide margin.

Does it need a GPU? No. The GPU path (Vulkan) mainly accelerates prompt processing — a Radeon 780M read prompts 5.2× faster than CPU-only. Decode speed comes from RAM and the SSD. JANAS_GPU=0 runs everything on CPU.

Can it run Llama, Mistral, or DeepSeek models? No. It runs a fixed catalog of 14 converted Qwen and Gemma models, distributed pre-converted. Arbitrary GGUF files don’t load.

Will this work on my 16GB laptop? Unknown, by the developer’s own admission — the memory thresholds were tuned on 32 GB, and 16 GB machines are a stated testing gap. The smaller catalog models (Gemma 4 E4B runs in ~3 GB) will work; the 80B is a gamble.

How does it compare to Strata? Strata targets a 12 GB+ discrete GPU plus 64 GB RAM and one 125B model family at 2-bit; Janas-LLM targets no GPU, 32 GB RAM, and tops out at the 80B at Q4. Different hardware, different ceiling — our Strata guide covers the other branch.

Is it safe to run? It’s GPL-3.0 open source you can audit, but treat it as beta: it can swap-storm your desktop (use --cache), and one GPU run caused a driver reset. Benchmarks before long sessions.

Sources

• Janas-LLM repository (README, license, catalog) — GitHub
• Janas-LLM benchmark tables and methodology — GitHub wiki
• Janas-LLM hardware notes (tested GPUs, OS support, tester reports) — GitHub
• Qwen3-Next-80B-A3B-Instruct GGUF files — Hugging Face / Unsloth
• Samsung 990 Pro 2TB back to $390 at Amazon — TechRadar
• Samsung 990 Pro 2TB $389.99 Amazon deal (in stock Oct 2, 2026) — Digital Citizen

Last updated October 14, 2026. Prices and specs change; verify current rates before purchasing. Benchmark figures are the Janas-LLM project’s own, measured on a single machine — expect different numbers on different hardware.

Recommended Gear

Products linked in this article:

• Samsung 990 Pro 2TB — the NVMe speed tier Janas-LLM’s benchmarks assume (~$370–$390, October 2026)
• Used RTX 3090 24GB — the speed-first alternative: 107 tok/s on Qwen3.6-35B-A3B ($1,399–$1,450 used, October 2026)

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe