Serge
Freea chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!
FreeFree tier
Inputs: textOutputs: text
About Serge
Serge is a free, open-source web interface for running LLaMA-based language models locally using llama.cpp. It is fully Dockerized for easy deployment and provides an API via FastAPI with LangChain integration. The frontend is built with SvelteKit and uses Redis for chat history storage. No API keys or cloud dependencies are required—everything runs self-hosted. The project is licensed under MIT and Apache-2.0 and was created by Nathan Sarrazin and contributors.
Key Features
Self-hosted – no API keys or cloud dependencies required
Fully Dockerized with a single Docker command or docker-compose
SvelteKit frontend for a modern chat interface
Redis for storing chat history and parameters
FastAPI + LangChain API wrapping llama.cpp calls via Python bindings
Built-in API documentation available at /api/docs
Configurable via environment variables (database, JWT, session expiry)
Open-source under MIT and Apache-2.0 licenses
Supports running Alpaca and other LLaMA family models
Pros & Cons
Pros
- Completely self-hosted, ensuring data privacy
- No API keys or subscription fees required
- Simple Docker deployment with pre-configured setup
- Uses efficient llama.cpp backend for CPU inference
- LangChain integration allows extensibility and prompt chaining
- Free and open-source with permissive licenses
Cons
- Project is archived and no longer maintained
- Requires significant RAM to run larger models (e.g., LLaMA-30B)
- Limited to LLaMA/Alpaca model family – not compatible with other architectures
- Docker knowledge needed for deployment and configuration
- No cloud-hosted version; must be run on own hardware
Best For
Running LLaMA or Alpaca language models locally for private chatSelf-hosted AI assistant without reliance on third-party APIsDevelopment and testing of LangChain-powered chatbotsEducational experimentation with local LLMsPrivacy-focused LLM applications
FAQ
How do I run Serge?
Use the Docker command: docker run -d --name serge -v weights:/usr/src/app/weights -v datadb:/data/db/ -p 8008:8008 ghcr.io/serge-chat/serge:latest. Then visit http://localhost:8008.
Is Serge free to use?
Yes, Serge is completely free and open-source software licensed under MIT and Apache-2.0.
What models does Serge support?
Serge uses llama.cpp to run LLaMA-based models, including Alpaca. It is designed for the LLaMA family and may not work with other model architectures.
Does Serge require an API key?
No, Serge is fully self-hosted and does not require any API keys. All processing is done locally.
How can I customize Serge’s settings?
Serge can be configured via environment variables such as SERGE_DATABASE_URL, SERGE_JWT_SECRET, and SERGE_SESSION_EXPIRY. See the README on GitHub for details.