semantic-router

vLLM Semantic Router is an open-source, system-level intelligent router designed for mixture-of-models (MoM) deployments across cloud, data center, and edge environments. It enables signal-driven routing with 16 signal families to classify intent, safety, and domain signals, then route each request to the most appropriate model. The router offers a drop-in OpenAI-compatible API, deployable as an Envoy ExtProc or locally via vllm-sr, requiring no client changes for existing integrations. It supports 12 configurable routing algorithms — from rule-based to reinforcement learning — and provides observability and security policy enforcement. Backward compatible with heterogeneous model pools (closed models like Claude, ChatGPT, Gemini and open models like Mistral, DeepSeek, etc.), it acts as a meta model that optimizes cost, privacy, and policy for enterprise AI.