LiteLLM Cache
FreeExact + Semantic
FreeFree tier
About LiteLLM Cache
LiteLLM's caching system stores and reuses LLM responses to save costs and reduce latency. It supports multiple backends including in-memory, disk, Redis, Qdrant semantic cache, Redis semantic cache, Valkey semantic cache, S3 bucket, GCS bucket, and virtual key authentication cache. Caching can be configured via a YAML config file with options for namespaces, TTL, and Redis cluster support.
Key Features
In-memory cache
Redis cache
Qdrant semantic cache
Redis semantic cache
Valkey semantic cache
S3 bucket cache
GCS bucket cache
Virtual key authentication cache (Redis)
Configurable TTL and namespaces
Redis cluster support
Pros & Cons
Pros
- Supports multiple caching backends including in-memory, disk, Redis, and cloud storage
- Includes semantic caching for identifying similar queries
- Open source and free to use
- Configurable TTL and namespaces for organized caching
- Redis cluster support for high availability
- Virtual key authentication caching reduces database load
Cons
- Default in-memory cache is worker-local, not shared across replicas (requires Redis for shared cache)
- Requires external services (Redis, Qdrant, etc.) for advanced caching features
- Configuration can be complex with multiple backend options
Best For
Reduce LLM API costsReduce response latency for repeated requestsCache virtual key authentication to avoid database queriesSemantic caching for similar queries
FAQ
How do I enable caching in LiteLLM?
Add `cache: true` under `litellm_settings` in your config.yaml file. Optionally configure `cache_params` for backends like Redis.
What caching backends does LiteLLM support?
LiteLLM supports in-memory, disk, Redis, Qdrant semantic cache, Redis semantic cache, Valkey semantic cache, S3 bucket, GCS bucket, and virtual key authentication cache (Redis).
How do I set up a Redis cluster for caching?
Configure `redis_startup_nodes` in the config.yaml or set the `REDIS_CLUSTER_NODES` environment variable with a JSON array of host/port objects.