TL;DR: audio.cpp is a pure C++/ggml engine that runs 20 open audio models — TTS, STT, voice conversion, music — with zero Python, hitting 1.8x–5x the speed of the reference PyTorch paths on the same CUDA GPU. The models are tiny (0.6B–7B), so this is not a VRAM problem: almost any modern 8GB card generates speech many times faster than real time. Buy for the software, not the silicon.

	audio.cpp (local C++)	Python / PyTorch reference	Cloud TTS API
Best for	Fast, private, offline speech on hardware you own	Research, fine-tuning, one-off scripts	Zero-setup, low volume
Speed	1.8x–5x faster than Python; up to 48x real-time	Baseline (heavy runtime overhead)	Network-bound
Cost	GPU electricity only (~$0.02/hr)	Same GPU, slower	Per-character billing
The catch	v0.1, build-from-source, quant support uneven	Slow startup, Python dep hell	Data leaves your machine

Honest take: If you already have any RTX 30-series or newer with 8GB+, audio.cpp is the fastest way to run local TTS and voice cloning today — and the model you pick matters far more than the card. Don’t buy a GPU for this; a used RTX 3090 is wild overkill, and a RTX 4060 Ti or even a laptop chip clears real time with room to spare.

What audio.cpp actually is

audio.cpp, released by ShugoAI as v0.1.0 on June 25, 2026 (with model waves through July 3), is to audio what llama.cpp is to text: an all-in-one inference engine written in C++ on top of ggml, with no Python dependency at runtime. One binary loads any supported model, runs on CPU, CUDA, Vulkan, or Metal, and speaks a common CLI. It ships under the Apache 2.0 license, so you can build it into a commercial product without a lawyer on speed-dial.

The June/July 2026 releases cover 20 models across seven task types — this is the number people keep getting wrong. Early write-ups said “12,” but the release branch actually documents:

• Text-to-speech: Chatterbox, MioTTS, OmniVoice, PocketTTS, Qwen3-TTS, Vevo2, VibeVoice, VoxCPM2
• Music generation: ACE-Step, HeartMuLa, Stable Audio 3
• Voice conversion: SeedVC
• Speech recognition (STT): Citrinet, Qwen3-ASR
• Voice activity detection (VAD): MarbleNet, Silero VAD
• Source separation: HTDemucs, Mel-Band RoFormer
• Codec: MioCodec

More models (Kokoro-TTS, MOSS-TTS, Higgs Audio, Parakeet-TDT) are in the testing pipeline. The unifying idea is that all of these are small transformer or flow-matching models that a general C++ runtime can serve efficiently, the same way llama.cpp collapsed a dozen incompatible LLM codebases into one.

The real speedups — and where they come from

The headline claim is “up to 5x faster than Python,” and unlike most vendor benchmarks, this one is measured honestly: audio.cpp and the matching Python reference paths were timed on the same CUDA setup, so it’s a like-for-like comparison, not a rigged demo. From the release benchmarks:

Model (one-shot)	Speedup vs Python	Wall-time saved
Vevo2	5.03x	80.1%
PocketTTS	3.68x	72.8%
MioTTS	2.73x	63.4%
MOSS-TTS	2.33x	57.1%
Qwen3-TTS	1.83x	45.3%
VibeVoice	1.40x	28.8%

On long-form passages (a 6,026-character block), the gap widens for the streaming-friendly models: PocketTTS hits 3.15x, Qwen3-TTS 3.06x, and VibeVoice 2.86x. In a long-lived session where the model stays warm, Qwen3-TTS reaches 2.74x with 63.5% less wall time.

Most of that win is not raw kernel speed — it’s the elimination of Python. PyTorch’s import time, the CUDA context spin-up, per-call graph tracing, and the object churn of a Python inference loop dominate short synthesis jobs. A compiled C++ engine with persistent CUDA graphs (on by default via -DENGINE_ENABLE_CUDA_GRAPHS=ON) pays none of that tax. That’s exactly why the biggest speedups show up on the smallest, fastest models: when the actual math takes 40ms, shaving a 200ms Python overhead is a 5x win.

The number that matters more for real use is the real-time factor (RTF) — seconds of compute per second of audio, where lower is better. audio.cpp’s measured one-shot RTFs:

• PocketTTS: 0.032 RTF — 31x faster than real time
• Kokoro-TTS: 0.063 RTF (15.9x)
• MOSS-TTS: 0.101 RTF (9.9x)
• OmniVoice: 0.146 RTF (6.8x)
• Chatterbox: 0.252 RTF (4.0x)

On long-form runs PocketTTS drops to 0.021 RTF — 48x faster than real time. Translation: a full minute of speech renders in a bit over a second. For any interactive assistant or narration pipeline, latency is a solved problem here.

The counterintuitive part: this is not a VRAM problem

Here’s where the “which GPU” question gets flipped on its head. Every buying guide on this site — from the best local models by VRAM tier to the used 3090 value analysis — hammers the same point for LLMs: VRAM decides what runs, bandwidth decides how fast. Audio models break that rule because they are tiny.

Look at the parameter counts audio.cpp actually serves:

• Qwen3-TTS: 0.6B and 1.7B
• Qwen3-ASR: 0.6B
• VibeVoice: 1.5B and 7B
• HeartMuLa: 3B
• Higgs Audio v3: 4B

Qwen3-TTS 1.7B in FP16 needs roughly 4.2 GB of VRAM for inference; the 0.6B variant needs meaningfully less. Even the largest audio model in the lineup (7B VibeVoice) fits comfortably in a 12GB card. There is no 42GB wall like a 70B LLM, no multi-GPU rig, no CPU-offload gymnastics. An 8GB GPU runs the entire catalog with headroom.

So the LLM buying logic doesn’t transfer. What you’re really choosing is a compute path — CUDA vs Metal vs CPU — and any GPU with enough VRAM to hold a 5GB model at once will saturate these workloads.

Which GPU you actually need

Because the models are small, the useful comparison isn’t “how much VRAM” — it’s “does the card clear real time comfortably, and how much thermal/noise/power am I paying for headroom I won’t use.”

Measured Qwen3-TTS 1.7B numbers from the community line up with the picture:

• RTX 4090, 1.7B, no FlashAttention: RTF 0.95, ~145ms latency. With FlashAttention 2: RTF 0.65, ~97ms — a 46% throughput bump and a third off latency.
• RTX 3090, 0.6B: RTF 0.52–0.68 (comfortably real time). The 1.7B on a 3090 sits around RTF 0.95–1.26 — right at the real-time line without FlashAttention, which is why the 0.6B model is the smart pick if you’re latency-sensitive on older silicon.

FlashAttention 2 is the single biggest free lever here: 30–40% faster and 20–25% less VRAM on any Ampere-or-newer card (RTX 30xx/40xx/50xx, A100, H100). Turn it on.

For a practical recommendation:

Your situation	Card	Why
Already have a gaming GPU	Whatever you own (8GB+)	Every audio.cpp model fits; you’re done
Building a dedicated box	RTX 4060 Ti 16GB or RTX 5060 Ti 16GB	Cheapest new 16GB, low power, silent, more than enough
Want to also run LLMs	RTX 3090 24GB used	24GB earns its keep on 30B LLMs, not on TTS
Mac user	Apple Silicon (Metal build)	Runs natively; see the M5 Max analysis

Spending $1,000+ on a card for TTS makes no sense. The audio models don’t need it, and audio.cpp’s whole reason to exist is that it makes modest hardware fast. If you’re buying a GPU that will also run local LLMs and image generation, then the usual VRAM-first logic applies — but that’s a decision the LLM side makes, not the audio side.

Building it: the flags that matter

audio.cpp builds from source. The compute backend is chosen at configure time:

# CUDA (NVIDIA)
cmake -S . -B build -DENGINE_ENABLE_CUDA=ON
cmake --build build --config Release

# Vulkan (cross-vendor, incl. AMD)
cmake -S . -B build -DENGINE_ENABLE_VULKAN=ON

# Metal (Apple Silicon)
scripts/build_metal.sh --target audiocpp_cli

CPU inference is the default and stays enabled unless you pass -DENGINE_ENABLE_NATIVE_CPU=OFF; OpenMP and CUDA Graphs are both on by default and you should leave them that way. The one build gotcha is the same one that trips up llama.cpp users: if you configure without the CUDA flag, you get a working-but-slow CPU binary and wonder why your GPU sits idle. Confirm your build linked CUDA before benchmarking.

If you don’t own a suitable GPU and want to test the larger models (VibeVoice 7B, music generation) before committing to hardware, spinning up a cloud GPU by the hour is the sane move — a single RTX 4090 or A100 pod on RunPod costs a couple dollars an hour and lets you profile RTF on your actual scripts before you buy anything.

Quantization: mostly not worth it here

For LLMs, quantization is the whole game — it’s how a 70B model squeezes onto a 24GB card. For audio.cpp, it barely moves the needle, and the release notes are refreshingly blunt about it. Measured q8_0 results:

• Qwen3-TTS q8_0: ~3.8% faster, with 25% lower peak VRAM and 3.7% lower peak RAM
• Vevo2 q8_0: ~3.6% faster

The docs explicitly warn that “quantization may not be safe on every path even when a model parser accepts the option,” that “output quality can drop a lot,” and that “the performance gain may be minor relative to that quality risk.” Given that FP16 already fits every model in 8GB, there’s little reason to quantize a TTS model unless you’re chasing that 25% VRAM cut on a truly tight card. Run the full-precision weights and spend your tuning effort on FlashAttention and picking the right model instead.

Where audio.cpp beats ComfyUI-based TTS

The site already covers MOSS-TTS voice cloning inside ComfyUI, which is the right tool if you want a visual node graph and you’re already living in the image-gen ecosystem. audio.cpp is the opposite philosophy: a headless engine you script or wire into a backend. It wins when you want a low-latency HTTP TTS endpoint, an offline assistant, or a batch narration job — anything where a 200ms Python overhead per call and a 30-second cold start actually hurt. The 48x-real-time PocketTTS number isn’t a lab curiosity; it’s what makes streaming TTS feel instant in a real app.

FAQ

Is audio.cpp really 5x faster than PyTorch? For Vevo2 in one-shot mode, yes — 5.03x measured on the same CUDA GPU. Other models range from 1.4x (VibeVoice) to 3.7x (PocketTTS). Most of the win is eliminating Python’s runtime overhead, so short jobs benefit most.

Do I need an expensive GPU? No. The models are 0.6B–7B parameters. Qwen3-TTS 1.7B uses ~4.2GB VRAM in FP16. Any RTX 30-series or newer with 8GB clears real time. The card matters far less than which model you run.

Does it run without an NVIDIA GPU? Yes. It builds for Metal (Apple Silicon), Vulkan (AMD and cross-vendor), and plain CPU. CPU is the default backend. The 5x figures are CUDA-measured, but Metal and Vulkan paths exist.

Should I quantize the models? Usually not. q8_0 buys only ~3.6–3.8% speed and 25% VRAM, and the docs warn quality can drop noticeably. Full precision already fits in 8GB, so leave it alone unless VRAM is genuinely tight.

Is it production-ready? It’s v0.1.0 (June–July 2026) and Apache 2.0 licensed, so the license is clean, but treat it as early software: build from source, expect rough edges, and validate quality on your own text before shipping.

How does it compare to Qwen3-TTS’s own Python code? audio.cpp runs the same Qwen3-TTS weights ~1.8x faster one-shot (up to 3.06x long-form) with no Python. Qwen3-TTS itself is Apache 2.0, 0.6B/1.7B, 10 languages, ~97ms end-to-end latency on a 4090 with FlashAttention 2.

Sources

• audio.cpp — pure C++/ggml audio inference engine (GitHub, 0xShug0)
• audio.cpp LICENSE — Apache 2.0, ShugoAI LLC 2026
• Qwen3-TTS — open-source TTS series (GitHub, QwenLM)
• Qwen3-TTS-12Hz-1.7B-CustomVoice model card (Hugging Face)
• Qwen3-TTS 1.7B VRAM & cheapest GPU (Spheron GPU recommender)
• Qwen3-TTS Performance Benchmarks and Hardware Guide 2026
• Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation (arXiv 2508.16332)
• Pocket TTS (Kyutai Labs)

Last updated July 7, 2026. audio.cpp is v0.1.0 and moving fast; benchmarks and model support change between releases — verify current numbers before building.

Recommended Gear

Only products actually linked above, none of which you need to buy for TTS — any 8GB+ GPU you already own runs everything:

• RTX 4060 Ti 16GB — cheapest new 16GB card, low power, silent, more than enough for every audio.cpp model
• RTX 5060 Ti 16GB — current-gen equivalent with GDDR7 bandwidth
• Used RTX 3090 24GB — overkill for audio, but the pick if the same box also runs 30B LLMs