Turn any SIP call into a realtime AI voice agent (OpenAI Realtime / Deepgram/Gemini Live)

SIP-to-AI

Why this project exists

Most voice agent frameworks:

• require WebRTC
• rely on heavy dependencies (C / ffmpeg)
• are not telephony-native

This project:

• pure Python SIP + RTP (no C deps)
• direct bridge to realtime AI
• built for call center / telephony scenarios

SIP-to-AI — stream RTP audio from FreeSWITCH / OpenSIPS / Asterisk directly to end-to-end realtime voice models:

• ✅ OpenAI Realtime API (gpt-realtime GA)
• ✅ Deepgram Voice Agent
• ✅ Gemini Live (Gemini 2.5 Flash)

Simple passthrough bridge: SIP (G.711 μ-law @ 8kHz) ↔ AI voice models. OpenAI and Deepgram support native G.711, Gemini requires PCM16 resampling (8kHz ↔ 16kHz/24kHz).

Quick Start (OpenAI Realtime)

Prerequisites: Python 3.12+, UV package manager

Pure Python, No External Dependencies: This project uses a pure Python asyncio implementation of SIP+RTP. No C libraries or compilation required!

1. Install dependencies:

   git clone <repository-url>
   cd sip-to-ai
   uv venv && source .venv/bin/activate
   uv sync
   

2. Configure environment:

   cp .env.example .env
   

   Edit .env with your OpenAI API key:

   # AI Service
   AI_VENDOR=openai
   OPENAI_API_KEY=sk-proj-your-key-here
   OPENAI_MODEL=gpt-realtime
   
   # Agent prompt
   AGENT_PROMPT_FILE=agent_prompt.yaml
   
   # SIP Settings (userless account - receive only)
   SIP_DOMAIN=192.168.1.100
   SIP_TRANSPORT_TYPE=udp
   SIP_PORT=6060
   

   Optional: Create agent_prompt.yaml for custom agent personality:

   instructions: |
     You are a helpful AI assistant. Be concise and friendly.
   
   greeting: "Hello! How can I help you today?"
   

3. Run the server:

   uv run python -m app.main
   

   The server will listen on SIP_DOMAIN:SIP_PORT for incoming calls. Each call creates an independent OpenAI Realtime WebSocket connection.

4. Make a test call:

   # From FreeSWITCH/Aste