Description
The real obstacle to running a speech model locally has never been the GPU. It is the third conda environment, the torch build that does not match your CUDA, the model that needs transformers 4.38 next to the one that needs 4.44, the twenty-minute requirements install that ends in an ImportError. By the time the environment works, the curiosity is gone.
audio.cpp replaces that path. It is a pure C++ audio inference engine built on ggml, with no Python anywhere in the inference core. Text to speech, speech recognition, voice cloning, voice conversion, source separation, diarization, VAD and music generation — capabilities that normally live in dozens of unrelated Python repositories — share one runtime, one CLI and one API server here.
More than 60 model families and 85 variants are already wired in, including Qwen3-TTS, Qwen3-ASR, Fish Audio, CosyVoice3, FireRedTTS3 and MiniMax Music 3, all loadable as GGUF. Performance is the point of the project: the published comparisons show several TTS paths running 1.8x to 8x faster than their Python reference implementations, with end-to-end latency down 45% to 85%.
One caveat worth stating plainly: CUDA is where the optimisation effort goes. HIP/ROCm, Vulkan, Metal and CPU-only backends all work, but both performance and model coverage trail the NVIDIA path, so check your hardware before committing.
One runtime for the whole pipeline: TTS, ASR, voice cloning, voice conversion, diarization, VAD, source separation, forced alignment and music generation share a single framework instead of one environment per model.
No Python dependency: The inference core is C++ on ggml and builds to executables and a server, so deployment carries no interpreter and no package tree to conflict.
GGUF quantisation: Every released family loads from GGUF; measured Q8 packages run up to about 1.53x faster and cut peak VRAM by roughly 37% on paths such as Higgs Audio, Fish Audio and Voxtral.
WebUI and Arena comparison: A native web interface for trying models in the browser, plus an Arena tab that queues several models or quantisation variants against one shared input and lays the results out side by side with metrics.
Unified API server: CLI and HTTP server share the same entry points, so a voice assistant, a dubbing tool or a batch script all talk to the same interface, with experimental JSON pipelines for multi-step workflows.
Many backends, many platforms: CUDA, HIP/ROCm, Vulkan, Metal and CPU builds across Windows, Linux and macOS, covering NVIDIA, AMD, Apple Silicon and machines with no GPU at all.
Audio utilities included: Denoise, enhancement, resampling and STFT/ISTFT are part of the framework, so a production pipeline does not need a separate stack of audio libraries.
Parity testing against Python: The project ships parity tooling that compares output against the Python reference implementations, so a port has to match results rather than merely run.
audio.cpp replaces that path. It is a pure C++ audio inference engine built on ggml, with no Python anywhere in the inference core. Text to speech, speech recognition, voice cloning, voice conversion, source separation, diarization, VAD and music generation — capabilities that normally live in dozens of unrelated Python repositories — share one runtime, one CLI and one API server here.
More than 60 model families and 85 variants are already wired in, including Qwen3-TTS, Qwen3-ASR, Fish Audio, CosyVoice3, FireRedTTS3 and MiniMax Music 3, all loadable as GGUF. Performance is the point of the project: the published comparisons show several TTS paths running 1.8x to 8x faster than their Python reference implementations, with end-to-end latency down 45% to 85%.
One caveat worth stating plainly: CUDA is where the optimisation effort goes. HIP/ROCm, Vulkan, Metal and CPU-only backends all work, but both performance and model coverage trail the NVIDIA path, so check your hardware before committing.
Features
One runtime for the whole pipeline: TTS, ASR, voice cloning, voice conversion, diarization, VAD, source separation, forced alignment and music generation share a single framework instead of one environment per model.
No Python dependency: The inference core is C++ on ggml and builds to executables and a server, so deployment carries no interpreter and no package tree to conflict.
GGUF quantisation: Every released family loads from GGUF; measured Q8 packages run up to about 1.53x faster and cut peak VRAM by roughly 37% on paths such as Higgs Audio, Fish Audio and Voxtral.
WebUI and Arena comparison: A native web interface for trying models in the browser, plus an Arena tab that queues several models or quantisation variants against one shared input and lays the results out side by side with metrics.
Unified API server: CLI and HTTP server share the same entry points, so a voice assistant, a dubbing tool or a batch script all talk to the same interface, with experimental JSON pipelines for multi-step workflows.
Many backends, many platforms: CUDA, HIP/ROCm, Vulkan, Metal and CPU builds across Windows, Linux and macOS, covering NVIDIA, AMD, Apple Silicon and machines with no GPU at all.
Audio utilities included: Denoise, enhancement, resampling and STFT/ISTFT are part of the framework, so a production pipeline does not need a separate stack of audio libraries.
Parity testing against Python: The project ships parity tooling that compares output against the Python reference implementations, so a port has to match results rather than merely run.
