Description
The awkward part of talking to a voice assistant: you wait for it to finish, it cannot tell a joke from anger, and every reply has the same flat tone. Sophon-Realtime is an open family of full-duplex voice models that keeps listening while you talk, hears your tone, and answers with a voice that laughs or sighs when the moment calls for it.
It drafts its reply as you speak, stops and answers your new question if you cut in, and gets first audio out in as little as 0.31 seconds. On public benchmarks its tone understanding ranks first, picking up emotion, emphasis, sarcasm, laughter and sighs straight from the voice. The laughs, sighs and breaths in its replies are decided by the model from the conversation and actually performed rather than read aloud as tags. Code and weights are Apache-2.0 and free for commercial use.
Three builds: Tag for expressiveness (companions, characters), Flash for speed (customer service, in-car), and E2E as a research preview, all sharing one 32B brain.
Full-duplex conversation: thinks while it listens and stops instantly when interrupted.
Multilingual: Chinese, English, Italian, German, Spanish, French and Portuguese out of the box, with Japanese and Russian on Flash.
Tool calling: works with a larger backend model and is compatible with the OpenAI GPT-Live interface format.
One GPU, one command: Tag and Flash serve through vLLM-Omni on a single 80 GB GPU, with about 16 and 40 concurrent users respectively.
It drafts its reply as you speak, stops and answers your new question if you cut in, and gets first audio out in as little as 0.31 seconds. On public benchmarks its tone understanding ranks first, picking up emotion, emphasis, sarcasm, laughter and sighs straight from the voice. The laughs, sighs and breaths in its replies are decided by the model from the conversation and actually performed rather than read aloud as tags. Code and weights are Apache-2.0 and free for commercial use.
Features
Three builds: Tag for expressiveness (companions, characters), Flash for speed (customer service, in-car), and E2E as a research preview, all sharing one 32B brain.
Full-duplex conversation: thinks while it listens and stops instantly when interrupted.
Multilingual: Chinese, English, Italian, German, Spanish, French and Portuguese out of the box, with Japanese and Russian on Flash.
Tool calling: works with a larger backend model and is compatible with the OpenAI GPT-Live interface format.
One GPU, one command: Tag and Flash serve through vLLM-Omni on a single 80 GB GPU, with about 16 and 40 concurrent users respectively.
