Description
Replacing OpenAI with DeepSeek or Llama on your own servers means fiddling with deployment, quantization and API compatibility. OpenLLM from BentoML serves any open-source LLM as an OpenAI-compatible API endpoint.
One command starts the server with a chat UI, deployable to BentoCloud and other clouds.
One command:Serve fast.
OpenAI-compatible:Drop-in.
Models:DeepSeek, Llama and more.
Chat UI:Included.
One command starts the server with a chat UI, deployable to BentoCloud and other clouds.
Features
One command:Serve fast.
OpenAI-compatible:Drop-in.
Models:DeepSeek, Llama and more.
Chat UI:Included.
