When to Retrieve During Reasoning

Analysis

Why This Paper Matters

Large reasoning models like DeepSeek-R1 and OpenAI o1 generate extended chains of thought, but current RAG systems are fundamentally misaligned because they provide context before reasoning begins, while reasoning models need evidence injection during multi-step inference. This paper addresses a critical gap: the timing of retrieval. By introducing a reasoning-aware retrieval framework, ReaLM-Retrieve, the authors show that injecting evidence at step-level uncertainty points significantly improves answer quality while reducing unnecessary retrieval calls. This is particularly important for multi-hop QA tasks where reasoning chains are long and knowledge gaps are intermittent.

The significance extends beyond QA benchmarks. As reasoning models become more prevalent in production systems, the ability to efficiently integrate external knowledge without disrupting the reasoning process is crucial. ReaLM-Retrieve's approach of learning when to retrieve, rather than retrieving at fixed intervals or only at the start, could influence how future RAG systems are designed for complex reasoning tasks.

Technical Contributions

• Step-level uncertainty detector: Unlike prior work that detects uncertainty at token or sentence level, this detector identifies knowledge gaps at the granularity of reasoning steps, which aligns better with the structure of multi-step reasoning chains.
• Retrieval intervention policy: A learned policy determines when external evidence maximally benefits ongoing reasoning, avoiding unnecessary retrievals that can disrupt the reasoning flow.
• Efficiency-optimized integration: Reduces per-retrieval overhead by 3.2x compared to naive integration, making the framework practical for real-time applications.
• Evaluation on three multi-hop QA benchmarks: MuSiQue (2-4 hops), HotpotQA, and 2WikiMultiHopQA, demonstrating robustness across different reasoning complexities.

Results

ReaLM-Retrieve achieves on average 10.1% absolute improvement in answer F1 over standard RAG across the three benchmarks (range: 9.0-11.8%). On the challenging MuSiQue benchmark, it reaches 71.2% F1 with only 1.8 retrieval calls per question. Compared to fixed-interval approaches like IRCoT, it reduces retrieval calls by 47% while improving retrieval quality: 81.3% Recall@5 with consistently higher precision and MRR on supporting evidence. All improvements are statistically significant at p<0.01 using paired bootstrap.

Significance

This work establishes a new state-of-the-art efficiency-accuracy trade-off for reasoning-intensive retrieval tasks. By aligning retrieval timing with reasoning needs, ReaLM-Retrieve demonstrates that intelligent retrieval policies can both improve answer quality and reduce computational cost. This has broad implications for deploying RAG with large reasoning models in production, where latency and cost are critical. The framework's ability to reduce retrieval calls by nearly half while improving F1 suggests that many current RAG systems are over-retrieving, wasting resources. Future work could extend this approach to other modalities (e.g., code, images) and explore integration with reinforcement learning for further policy optimization.