Implement Custom LiteLlm for Google ADK with Tool Call Support

To implement a custom LiteLlm wrapper for the Google Agent Development Kit (ADK) that supports tool calls, you must propagate tool call information from your LLM endpoint's response into LiteLlm's ModelResponse object. The common mistake is only mapping message.content while ignoring the tool_calls array. The accepted solution involves subclassing BaseLlm directly from google.adk.models.base_llm and manually constructing LlmResponse objects that include types.Part instances for both text and function calls.

The Full Answer

Understanding the Problem

The original code from Source 1 creates a custom LiteLlm class that inherits from litellm.CustomLLM. It works for simple text generation but fails when the agent needs to call a tool. The root cause is in the process_request method: it only extracts content from the API response and maps it into a litellm.Message. The tool_calls field, which the API may return when the model decides to invoke a tool, is completely ignored. Without this field in the ModelResponse, LiteLlm and consequently the ADK agent never see that a tool should be called. The agent receives only text and proceeds as if no tool is needed, leading to the behavior described: "my tool never gets called."

Solution: Subclass BaseLlm Directly

The accepted solution from Source 2 takes a different approach. Instead of wrapping LiteLlm's CustomLLM, it subclasses BaseLlm from the ADK itself (google.adk.models.base_llm). This gives direct control over how the LLM response is parsed and how tool calls are represented. The key classes involved are:

• LlmRequest: Contains the incoming request, including contents (messages) and tools (tool definitions).
• LlmResponse: The response object that the ADK agent expects. It contains a Content object with a list of Part objects.
• types.Part: A part can be text (via Part.from_text()) or a function call (via types.Part(function_call=...)).

Step-by-Step Implementation

1. Create a Custom HTTP Client

First, define a minimal async HTTP client to communicate with your LLM endpoint. This client handles authentication and sends requests to a standard OpenAI-compatible chat completions endpoint.

import aiohttp
from typing import Any, Dict, Optional

class MyCustomClient:
    """Minimal HTTP client for a custom LLM endpoint."""

    def __init__(self, base_url: str, api_key: str):
        self.base_url = base_url.rstrip("/")
        self.api_key = api_key
        self.session: Optional[aiohttp.ClientSession] = None

    async def _get_session(self) -> aiohttp.ClientSession:
        if self.session is None or self.session.closed:
            self.session = aiohttp.ClientSession()
        return self.session

    async def close(self):
        if self.session and not self.session.closed:
            await self.session.close()

    async def generate(self, payload: Dict[str, Any]) -> Dict[str, Any]:
        session = await self._get_session()
        headers = {"Authorization": f"Bearer {self.api_key}"}
        url = f"{self.base_url}/v1/chat/completions"

        async with session.post(url, json=payload, headers=headers) as response:
            response.raise_for_status()
            return await response.json()

What this does: It creates a reusable session, sends a POST request to your endpoint with the payload (messages and tools), and returns the JSON response. The close() method ensures proper cleanup.

2. Implement the Custom LLM Class

Now, create the main class that inherits from BaseLlm. This class must implement the generate_content_async method, which is an async generator that yields LlmResponse objects.

import logging
from typing import AsyncGenerator
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_request import LlmRequest
from google.adk.models.llm_response import LlmResponse
from google.genai import types

logger = logging.getLogger(__name__)

class MyLiteLlm(BaseLlm):
    """Custom LLM implementation that supports tool calls."""

    def __init__(self, model: str, base_url: str, api_key: str, **kwargs):
        super().__init__(model=model, **kwargs)
        self._client = MyCustomClient(base_url, api_key)
        self._model = model

    async def generate_content_async(
        self, llm_request: LlmRequest, stream: bool = False
    ) -> AsyncGenerator[LlmResponse, None]:
        if not llm_request.contents:
            raise ValueError("LlmRequest must contain contents")

        payload = {
            "model": self._model,
            "messages": [c.model_dump(exclude_none=True) for c in llm_request.contents],
        }

        # optional: include tool declarations if present
        if llm_request.tools:
            payload["tools"] = [t.model_dump(exclude_none=True) for t in llm_request.tools]

        api_response = await self._client.generate(payload)

        # Convert response into ADK's LlmResponse, preserving tool calls
        parts = []
        for choice in api_response.get("choices", []):
            msg = choice.get("message", {})
            if "content" in msg and msg["content"]:
                parts.append(types.Part.from_text(text=msg["content"]))
            if "tool_calls" in msg:
                for tc in msg["tool_calls"]:
                    parts.append(
                        types.Part(
                            function_call=types.FunctionCall(
                                name=tc.get("function", {}).get("name", ""),
                                args=tc.get("function", {}).get("arguments", {}),
                            )
                        )
                    )

        llm_response = LlmResponse(
            content=types.Content(
                role=api_response.get("role", "model"),
                parts=parts,
            ),
            partial=False,
        )
        yield llm_response

    async def __aenter__(self):
        return self

    async def __aexit__(self, exc_type, exc_val, exc_tb):
        await self._client.close()

Key details:

• The payload dictionary is constructed from llm_request.contents. The model_dump(exclude_none=True) method serializes each content object into a dictionary, excluding any fields that are None. This ensures compatibility with standard OpenAI API formats.
• If llm_request.tools is present (a list of tool definitions), they are also serialized and included in the payload under the "tools" key. This tells your LLM endpoint that tools are available.
• After receiving the API response, the code iterates over choices. For each choice, it checks for both content and tool_calls in the message. Text content becomes a Part.from_text(). Each tool call becomes a types.Part with a function_call attribute containing the function name and arguments.
• The LlmResponse is constructed with a Content object containing all parts. The partial=False flag indicates this is a complete response (not streaming).
• The __aenter__ and __aexit__ methods enable use as an async context manager, ensuring the HTTP client is properly closed.

3. Use the Custom LLM with an ADK Agent

Once the class is defined, you can instantiate it and pass it to an ADK agent. The agent will call generate_content_async when it needs to generate a response, and tool calls will be handled automatically.

my_llm = MyLiteLlm(
    model="your-model-name",
    base_url="https://your-endpoint.com",
    api_key="your-api-key"
)

# Then pass my_llm to your ADK agent
# agent = YourAgent(llm=my_llm, ...)

When to Use Which Approach

• Use the BaseLlm subclass approach (Source 2) when you need full control over how tool calls are represented and your LLM endpoint returns tool calls in a standard format (e.g., OpenAI-compatible). This is the recommended solution for supporting tools.
• Use the litellm.CustomLLM approach (Source 1) only if you do not need tool call support and are only doing simple text generation. It is simpler but incomplete for agent workflows.