Description
Adding voice cloning to a project — dubbing, audiobooks, a voice assistant — usually hits a wall: most open TTS models are several gigabytes and will not fit a local GPU, or run too slowly for real time, or top out at a muffled 24kHz. Rarely all three at once.
LuxTTS aims to give you all three. It is a lightweight zipvoice-based voice cloning model that clones a voice from a reference clip and synthesises new speech, with cloning quality the project claims is on par with models ten times its size. It reaches 150x realtime on a single GPU and faster than realtime on CPU, and the whole model fits within 1GB of VRAM, so almost any local GPU can run it.
Audio quality is the other selling point: 48kHz output where most TTS models stop at 24kHz. Usage is simple — load the model, encode a reference clip with encode_prompt, call generate_speech — a few lines of Python, with an adjustable sampling step count (3-4 is the sweet spot for quality versus speed).
Run it locally, on Colab, or the HuggingFace Space. Apache 2.0, 5.3k stars.
High-quality voice cloning: Clone a voice from a reference clip and synthesise new speech, with cloning quality the project says rivals models ten times larger.
150x realtime speed: Up to 150x realtime generation on a single GPU and faster than realtime on CPU, suiting batch and live use.
48kHz output: Produces clear 48kHz speech, above the 24kHz common to most TTS models, for more natural audio.
Runs in 1GB VRAM: The whole model fits within 1GB of VRAM, so almost any local GPU can run it.
A few lines to integrate: Load the model, encode a reference with encode_prompt, synthesise with generate_speech — a few lines of Python.
Multiple run options: Local (CUDA/CPU/Mac MPS), Google Colab or the HuggingFace Space, for quick trials or real deployment.
LuxTTS aims to give you all three. It is a lightweight zipvoice-based voice cloning model that clones a voice from a reference clip and synthesises new speech, with cloning quality the project claims is on par with models ten times its size. It reaches 150x realtime on a single GPU and faster than realtime on CPU, and the whole model fits within 1GB of VRAM, so almost any local GPU can run it.
Audio quality is the other selling point: 48kHz output where most TTS models stop at 24kHz. Usage is simple — load the model, encode a reference clip with encode_prompt, call generate_speech — a few lines of Python, with an adjustable sampling step count (3-4 is the sweet spot for quality versus speed).
Run it locally, on Colab, or the HuggingFace Space. Apache 2.0, 5.3k stars.
Features
High-quality voice cloning: Clone a voice from a reference clip and synthesise new speech, with cloning quality the project says rivals models ten times larger.
150x realtime speed: Up to 150x realtime generation on a single GPU and faster than realtime on CPU, suiting batch and live use.
48kHz output: Produces clear 48kHz speech, above the 24kHz common to most TTS models, for more natural audio.
Runs in 1GB VRAM: The whole model fits within 1GB of VRAM, so almost any local GPU can run it.
A few lines to integrate: Load the model, encode a reference with encode_prompt, synthesise with generate_speech — a few lines of Python.
Multiple run options: Local (CUDA/CPU/Mac MPS), Google Colab or the HuggingFace Space, for quick trials or real deployment.
