Oct 28, 2026

Voxtral 3 for Local AI: Mistral Shipped No Weights — Here's What You Can Actually Run (2026)

By RunAIHome Team · 13 min read

voxtralmistralspeech-to-texttranscriptionlocal-aigpuhardware-guide

TL;DR: Voxtral 3, the speech family Mistral launched on October 20, 2026, is API-only — no weights, no license, nothing to download. The models you can run at home are the older open Voxtral checkpoints: Mini 3B and Small 24B (Apache 2.0, July 2025) and Realtime 4B (Apache 2.0, February 2026). A used RTX 3060 runs the Mini; the Small wants a 24GB card.

	Voxtral Mini 3B (local)	Voxtral Small 24B (local)	Voxtral 3 (API)
Best for	Transcription on any 6–12GB GPU	Transcription + audio Q&A on one card	Volume transcription, voice agents
Cost	GGUF Q4_K_M is a 2.47GB file; used RTX 3060 $260–$296	Q4_K_M ~14.3GB; used RTX 3090 $1,456–$1,558	$0.21–$0.24 per audio hour (batch)
The catch	~Whisper-level accuracy, not Voxtral 3 level	bf16 wants ~55GB; you’ll live at Q4/Q8	Your audio leaves the building

Honest take: If you already own a GPU with 6GB or more, run the open checkpoints tonight — the marginal cost of local transcription is about a penny per audio hour. If you’re buying hardware only to transcribe, the math is harder than you’d like: the API costs a quarter per hour, so a $1,500 card takes thousands of hours to pay off. Buy the card for privacy and unlimited volume, not savings.

Voxtral 3 hit the top of Hacker News this week, and the obvious question for anyone running a home lab is the one Mistral’s announcement never answers directly: can I run it on my own GPU? The answer is no — and that’s a meaningful break from how Mistral has treated this product line for the past fifteen months. Here’s what shipped, what’s still open, and what hardware actually runs the open parts.

What Mistral shipped on October 20 — and what it didn’t

Voxtral 3 is a family of four speech models plus a voice-agent SDK, all in public preview on Mistral’s platform (release coverage):

Model	Job	Price
Voxtral Mini Transcribe 3 (4B)	Batch speech-to-text	$0.0035/min
Voxtral Small Transcribe 3	Batch speech-to-text, higher accuracy	$0.004/min
Voxtral Realtime Transcribe 3	Streaming speech-to-text	$0.008/min
Voxtral Mini TTS 3	Text-to-speech	$13 per 1M characters

The accuracy numbers are genuinely strong. Small Transcribe 3 posts a 2.1% word error rate on the Artificial Analysis aggregate benchmark, ranking third overall among commercial transcription APIs (MarkTechPost). The realtime model scores 6.4% WER on a 35-language FLEURS evaluation at its 480ms latency setting — behind Gemini 3.5 Realtime Transcribe’s 4.2%, but priced well under it (AICoder). Transcription covers 50+ languages, and deployment options are Mistral’s EU-hosted cloud or an enterprise self-deployment that starts with a call to sales.

What’s missing is the thing that made Voxtral interesting to this site’s readers in the first place: no open weights were announced, no Hugging Face repos exist, and no license is published (MarkTechPost). The original Voxtral launch in July 2025 put both checkpoints on Hugging Face under Apache 2.0 the same day. Voxtral Realtime followed in February 2026, also Apache 2.0. Voxtral 3 breaks the pattern — at least at launch. Mistral has back-filled weights before (the TTS line went open five months after the API version), so this may change. Today, it’s an API product.

The open Voxtral stack you can run tonight

None of this makes your GPU less useful, because the previous generation is still on Hugging Face and still Apache 2.0. Four checkpoints matter:

Model	Released	License	What it does	Weights on disk
Voxtral Mini 3B 2507	Jul 2025	Apache 2.0	Transcription + audio understanding	GGUF Q4_K_M: 2.47GB
Voxtral Small 24B 2507	Jul 2025	Apache 2.0	Transcription + audio Q&A, summarization	GGUF Q4_K_M: ~14.3GB; bf16 ~48.5GB
Voxtral Mini 4B Realtime 2602	Feb 2026	Apache 2.0	Streaming transcription, 13 languages	~4B params
Voxtral 4B TTS 2603	Mar 2026	CC BY-NC 4.0	Text-to-speech	GGUF Q4_K: ~2.4GB

These aren’t consolation prizes. The 2507 models pair a Whisper large-v3 speech encoder with a Mistral LLM decoder (Ministral 3B for the Mini, Mistral Small 3.1 for the Small), per the Voxtral technical report. On LibriSpeech test-clean, Voxtral Small posts 1.53% WER against Whisper large-v3’s 1.84%, and Mistral’s FLEURS results have the Small ahead of Whisper across the tested languages. The honest caveat: those are the vendor’s own evaluations, and the Mini is roughly level with Whisper on several test sets rather than ahead.

The practical difference from plain Whisper is that a Voxtral model is also an LLM. You can hand it a 40-minute recording and ask “what did we decide about the deadline?” instead of transcribing first and feeding the text to a second model. Both 2507 models carry a 32K-token context — about 30 minutes of audio for transcription or 40 minutes for comprehension tasks in a single request.

The realtime checkpoint is the newest open piece: trained end-to-end for streaming with a configurable delay, with 480ms as the recommended setting, where the technical report claims accuracy on par with Whisper. Thirteen languages, ~4B parameters, sized for edge hardware.

VRAM math: which GPU tier runs what

Official requirements first, from Mistral’s model cards: ~9.5GB of GPU memory for Voxtral Mini 3B in bf16, ~55GB for Voxtral Small 24B in bf16/fp16. Take the 55GB seriously if you want full precision — that’s two 32GB cards or one workstation GPU, which is exactly the wrong hardware for a transcription box.

Quantization is what makes this consumer hardware territory. llama.cpp has supported Voxtral’s audio input since July 2025 (PR #14862 added it to the mtmd multimodal library), and both official-adjacent and community GGUFs exist — ggml-org publishes the Mini and bartowski covers the Small. The working map:

Your GPU	VRAM	What fits
RTX 3060 / 4060-class	8–12GB	Mini 3B at Q4–Q8 with room to spare; Realtime 4B
RTX 4060 Ti 16GB / RX 9070 XT	16GB	Small 24B Q4_K_M (14.3GB weights) — tight; short contexts only
Used RTX 3090 / 4090	24GB	Small 24B Q4_K_M comfortable, Q8 (~25.8GB) just misses
Strix Halo / Mac with 32GB+ unified	32GB+	Small 24B Q8_0

Two cautions on that table. The 16GB row is weights-only math — the mmproj audio encoder, KV cache, and a long audio context eat the remaining headroom fast, so treat 16GB as “it loads” rather than “it’s comfortable.” And speed estimates for the Small follow the bandwidth ceiling we use for every dense model: a 14.3GB Q4_K_M read on an RTX 3090’s 936 GB/s tops out around 65 tok/s decode, so expect real-world output in the 45–60 tok/s range. For transcription that’s far faster than you need — people speak at roughly 2–3 tokens per second — so the wait is dominated by audio-encoder prefill, not decode.

If your card is smaller than the model you want, check your exact combination in our VRAM calculator before buying anything.

Running it: one command, one trap

The shortest path is llama.cpp’s server, which pulls the quant and the audio projector together:

llama-server -hf ggml-org/Voxtral-Mini-3B-2507-GGUF:Q4_K_M

Expected startup tail:

srv  load_model: loading mmproj model ... mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf
main: server is listening on http://127.0.0.1:8080

From there, the OpenAI-compatible chat endpoint accepts input_audio content parts — send base64 WAV/MP3 and a “transcribe this” prompt. llama.cpp’s own docs still flag audio as experimental, and our testing matches that: for bulk, repeatable transcription jobs, faster-whisper remains the boring, reliable choice; Voxtral earns its place when you want transcription and understanding in one model.

The trap that will actually cost you an evening: the Realtime 2602 GGUFs on Hugging Face do not load in mainline llama.cpp. Community conversions like memoravox/Voxtral-Mini-4B-Realtime-2602-gguf target transcribe.cpp, a separate runtime built for the model’s dual-stream architecture, and the llama.cpp pull request that proposed native support (PR #20638) was closed without merging. If llama.cpp greets your download with an unknown-architecture error, that’s why. Fix: use the 2507 GGUFs in llama.cpp, and run the realtime model through transcribe.cpp or Mistral’s reference stack instead.

One real community data point on realtime-class speed: a TypeWhisper plugin benchmark measured the Realtime 4B at Q4_K_M on an RTX 4060 Ti via Vulkan at roughly real-time throughput with 9.6% WER — usable for live captions, too slow for snappy dictation (PR #723). CPU-only is rougher: a 4-core CPU chewed through an 11-second clip in 70 seconds at Q4_K on the Mini — about 6× slower than real time. Transcription boxes want a GPU, even a small one.

Local vs. the Voxtral 3 API: the actual money

Batch transcription on the API costs $0.0035–$0.004 per minute — $0.21–$0.24 per hour of audio. Realtime is $0.008/min, or $0.48 per audio hour.

Local marginal cost is electricity. A used RTX 3090 draws about 285W under inference load — $0.034 per wall-clock hour at $0.12/kWh (our 24/7 server power math) — and transcribes an hour of audio in 12–22 minutes with a Whisper-class encoder, which puts the power cost of a transcribed hour at roughly one cent. The API is ~20× more expensive per hour than local electricity.

But the card isn’t free, and this is where buying-for-transcription math gets honest:

Used RTX 3060 12GB at $260–$296 (October 2026 street price): runs the Mini 3B and faster-whisper with headroom. Against $0.20/hour saved, it breaks even around 1,300 hours of audio. A daily one-hour meeting pays it off in about four years — or one year if you’re transcribing four hours a day.
Used RTX 3090 24GB at $1,456–$1,558 (verified October 20, 2026): breaks even past 7,000 audio hours on transcription alone. Nobody should buy this card only to transcribe. It makes sense because it also runs every 24GB-class local LLM and the Small 24B’s audio-understanding mode — transcription becomes a free side quest.

The non-financial reasons are the real reasons: audio of your meetings, medical appointments, and calls is the single most privacy-sensitive data most people would ever upload, and local inference has no per-minute meter, no rate limits, and no network dependency.

What you give up by staying local

Three things, stated plainly. First, accuracy: Voxtral 3’s Small Transcribe posts 2.1% aggregate WER on third-party ranking; the open 2507 models are Whisper-class — excellent, but a generation behind on messy real-world audio. Second, the voice-agent SDK: the turn-taking, interruption handling, and tool-calling plumbing is an API product with no open equivalent you can pip install. Third, polish: diarization and word-level timestamps on the API are a parameter; locally they’re a WhisperX pipeline you assemble yourself.

If those three are worth more than a quarter per hour to you, use the API. For everything else, the open stack holds up — and for TTS, the local picture is covered in our audio.cpp guide and MOSS-TTS voice cloning walkthrough.

What to actually buy

Prices as of October 2026, all verified in the comparison above:

Your situation	The pick	Price	Where
Already own a 6GB+ GPU	Voxtral Mini 3B GGUF — run it tonight	$0	—
Want a cheap dedicated transcription box	Used RTX 3060 12GB	$260–$296	Check price
Want Small 24B + a real local-LLM card	Used RTX 3090 24GB	$1,456–$1,558	Check price
Want to test the workload before buying	Rented 3090	from $0.07/hr	Vast.ai
Volume transcription, no hardware appetite	Voxtral 3 API	$0.21–$0.24/audio hr	mistral.ai

FAQ

Q: Can I run Voxtral 3 locally at all?

A: No. As of late October 2026 there are no Voxtral 3 weights on Hugging Face and no published license — it’s API and enterprise self-deployment only. The models you can run locally are Voxtral Mini 3B and Small 24B (July 2025) and Voxtral Mini 4B Realtime (February 2026), all Apache 2.0.

Q: Will Mistral release Voxtral 3 weights later?

A: Unknown, but there’s precedent in both directions. The original Voxtral and the Realtime model shipped weights on day one; Voxtral Transcribe 2 stayed API-only, while the TTS line got an open (non-commercial) release months after its API debut. Watch the mistralai Hugging Face org rather than the press releases.

Q: Is local Voxtral better than Whisper large-v3?

A: On Mistral’s published benchmarks, the Small 24B beats Whisper large-v3 (1.53% vs 1.84% WER on LibriSpeech clean) and the Mini roughly matches it. The bigger difference is functional: Voxtral is also an LLM, so it can summarize and answer questions about audio directly. For pure bulk transcription on small hardware, faster-whisper is still the most mature pipeline.

Q: How much VRAM does Voxtral Small 24B need?

A: ~55GB in bf16 per Mistral’s model card — but the Q4_K_M GGUF is ~14.3GB, which runs comfortably on a 24GB card and loads (tightly) on 16GB. The Mini 3B needs ~9.5GB at bf16 and just 2.47GB at Q4_K_M, so nearly any modern GPU qualifies.

Recommended Gear

Used RTX 3060 12GB — $260–$296, the cheapest card that makes local transcription effortless
Used RTX 3090 24GB — $1,456–$1,558, runs Voxtral Small 24B Q4 plus the full 24GB local-LLM catalog

Sources

Last updated October 28, 2026. Prices and specs change; verify current rates before purchasing.

Was this article helpful?

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.