FlashRank
FreeLite Super-fast re-ranking for your search retrieval pipelines.
FreeFree tier
About FlashRank
FlashRank is an ultra-lite, super-fast Python library for adding re-ranking to existing search retrieval pipelines. It is based on state-of-the-art cross-encoders and LLMs, supporting both pairwise/pointwise and listwise reranking methods. The library runs on CPU, requires no PyTorch or Transformers, and offers models as small as ~4MB for low-cost, serverless-friendly deployments.
Key Features
Ultra-lite: no PyTorch or Transformers needed, runs on CPU, smallest model ~4MB
Super-fast: rerank speed depends on token count and model depth
Cost-conscious: lowest $ per invocation, small package size for serverless
Supports SoTA cross-encoders and LLMs: pairwise/pointwise and listwise rerankers
Multiple models available including multilingual and tinyBERT variants
Installation via pip, optional listwise support with flashrank[listwise]
Pros & Cons
Pros
- Extremely small model size (4MB) enables fast CPU inference
- No dependency on heavy frameworks like PyTorch or Transformers
- Low cost per invocation in serverless environments
- Supports both pairwise (pointwise) and listwise reranking methods
- Competitive performance for zero-shot reranking tasks
Cons
- Reranking speed depends on passage token count and model depth
- Listwise models require additional installation and have larger size (e.g., 4GB)
- Limited to specific pre-trained models; custom model addition requires pull request
- Only covers reranking stage, not full search or indexing
Best For
Re-ranking search results in RAG pipelines before feeding into LLMsImproving retrieval quality for real-time user-facing searchServerless deployments (e.g., AWS Lambda) due to low memory footprintLightweight reranking in on-premise or CPU-only environments
FAQ
What is FlashRank?
FlashRank is an ultra-lite, super-fast Python library for re-ranking search results using SoTA cross-encoders and LLMs. It is designed to be added to existing search retrieval pipelines.
Does FlashRank require a GPU?
No, FlashRank runs on CPU. It does not depend on PyTorch or Transformers, making it lightweight and suitable for serverless deployments.
How do I install FlashRank?
For pairwise/pointwise rerankers run 'pip install flashrank'. For listwise LLM based rerankers run 'pip install flashrank[listwise]'.
What models are supported?
Several models are supported including ms-marco-TinyBERT-L-2-v2 (default, ~4MB), ms-marco-MiniLM-L-12-v2 (~34MB), rank-T5-flan (~110MB), ms-marco-MultiBERT-L-12 (multilingual, ~150MB), rank_zephyr_7b_v1_full (~4GB), and others.
How can I make ranking faster?
Set the max_length parameter appropriately to accommodate your longest passage. Avoid using unnecessarily large max_length values as it negatively affects response time.