Haru-4B

Haru-4B

Full-duplex real-time spoken dialogue model

AIOfficial site

Description

Most voice assistants still work like walkie-talkies: they wait for you to finish, and if you cut in, the conversation falls apart. Haru-4B is a full-duplex spoken dialogue foundation model that listens and speaks at the same time, like a phone call. It says "mm-hmm" while you talk, stops when interrupted and picks up its turn naturally.

Every 133 ms it takes one frame of your audio and produces one frame of its own audio and text, all summed into a single position of the language model, so listening and speaking never wait on each other. The team reports about 65 ms end-to-end latency on a MacBook. It was pretrained on 40 million hours of real speech rather than LLM- or TTS-generated turn-based dialogue, so backchannels, overlaps and turn-taking come from real people. Nine languages: English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian and Korean. Apache 2.0, weights on Hugging Face. It is a pretrained base model meant for fine-tuning and research, not a finished assistant.

Features



Native full-duplex: processes incoming and outgoing speech together at 7.5 Hz, with no end-of-turn detection step.

One-step continuous audio: each frame is a 64-dimensional latent produced in one step by a small MLP head with MeanFlow sampling. No separate audio transformer, no discrete audio tokens.

Character and voice control: a text card sets the persona and a 3-second to 3-minute reference clip sets the voice. No voices ship with the repo; you upload or record your own.

Three inference stacks: PyTorch on an NVIDIA GPU with about 12 GB of memory, MLX on Apple silicon with a native macOS app, and a vLLM-Omni plugin for serving.

Web demo: one command starts a local server; pick a character, give it a voice and talk. --share prints a public HTTPS link for others to try.

Python API: load the model, prepare a reference voice and open a duplex session in a few lines.