Description
Fay is an open-source agent framework for digital humans. It does not render the face; it handles everything above the neck — listening, thinking, speaking, remembering, and reaching into business systems when needed. The front end can be Unity3D, UE5, a web page or a phone app, with Fay supplying the brain behind it.
Compared with cloud-dependent products, its practical difference is that the whole thing can run on your own machine. The language model, speech recognition and speech synthesis layers are each replaceable: point them at an OpenAI-compatible service, or swap in local models and run offline so nothing leaves the box. A memory system and MCP service management come with it, so the agent can decide which tool to call to get something done rather than only answering questions one at a time.
The licence is GPL-3.0. Commercial use is permitted, but this is strong copyleft: distribute a product built on it and that product must also be released under GPL-3.0 with source available. Anyone planning a closed-source commercial product should work through that first.
Swappable models: The LLM, ASR and TTS layers are not tied to any provider — use a cloud API, or replace all three with local models and run offline.
Tool calling: The agent works out for itself which tool to invoke or which interface to query, then shapes the result into an answer, rather than limiting itself to single-turn replies.
Memory: It remembers earlier conversation and supports a separate memory space per user, so multiple people do not bleed into each other.
MCP management: Connecting and managing MCP services is built in, which is how external capabilities get attached for the agent to use.
Persona configuration: The control panel sets name, gender, occupation, role and supplementary character notes, along with the wake word, wake mode, voice and sensitivity.
Many front ends: 2.5D, 3D, desktop, mobile and web avatars can all be driven, with established paths for Unity3D and UE5.
Self-hosted: The whole stack deploys to your own server or computer, with no dependence on one cloud vendor’s account and quota.
Compared with cloud-dependent products, its practical difference is that the whole thing can run on your own machine. The language model, speech recognition and speech synthesis layers are each replaceable: point them at an OpenAI-compatible service, or swap in local models and run offline so nothing leaves the box. A memory system and MCP service management come with it, so the agent can decide which tool to call to get something done rather than only answering questions one at a time.
The licence is GPL-3.0. Commercial use is permitted, but this is strong copyleft: distribute a product built on it and that product must also be released under GPL-3.0 with source available. Anyone planning a closed-source commercial product should work through that first.
Features
Swappable models: The LLM, ASR and TTS layers are not tied to any provider — use a cloud API, or replace all three with local models and run offline.
Tool calling: The agent works out for itself which tool to invoke or which interface to query, then shapes the result into an answer, rather than limiting itself to single-turn replies.
Memory: It remembers earlier conversation and supports a separate memory space per user, so multiple people do not bleed into each other.
MCP management: Connecting and managing MCP services is built in, which is how external capabilities get attached for the agent to use.
Persona configuration: The control panel sets name, gender, occupation, role and supplementary character notes, along with the wake word, wake mode, voice and sensitivity.
Many front ends: 2.5D, 3D, desktop, mobile and web avatars can all be driven, with established paths for Unity3D and UE5.
Self-hosted: The whole stack deploys to your own server or computer, with no dependence on one cloud vendor’s account and quota.
