Neura Market
RAGEN-2: Reasoning Collapse in Agentic RL (April 2026) logo

RAGEN-2: Reasoning Collapse in Agentic RL (April 2026)

Free

Identifies "template collapse" in agentic RL — models rely on fixed input-agnostic templates despite stable entropy; proposes mutual information (not entropy) as diagnostic for reasoning quality; Northwestern/Stanford/Microsoft; 49 HF likes

FreeFree tier
Type
Open Source
Visit Website GitHub (40 stars)

About RAGEN-2: Reasoning Collapse in Agentic RL (April 2026)

RAGEN-2 is a research paper that identifies a failure mode called 'template collapse' in multi-turn LLM agent RL training, where models rely on fixed input-agnostic reasoning templates despite stable entropy. The paper proposes using mutual information (MI) instead of entropy as a diagnostic for reasoning quality, introduces a family of MI proxies for online diagnosis, and explains template collapse via a signal-to-noise ratio (SNR) mechanism. It further proposes SNR-Aware Filtering to select high-signal prompts, improving input dependence and task performance across planning, math reasoning, web navigation, and code execution tasks.

Key Features

Identifies template collapse, a failure mode invisible to entropy and existing metrics
Decomposes reasoning quality into within-input diversity (entropy) and cross-input distinguishability (mutual information)
Introduces a family of mutual information proxies for online diagnosis of reasoning quality
Explains template collapse via a signal-to-noise ratio (SNR) mechanism where low reward variance weakens task gradients
Proposes SNR-Aware Filtering to select high-signal prompts using reward variance as a lightweight proxy

Pros & Cons

Pros
  • Reveals a failure mode not detectable by existing metrics like entropy
  • Mutual information correlates with final performance more strongly than entropy, offering a more reliable proxy
  • Proposes a practical filtering method (SNR-Aware Filtering) that consistently improves both input dependence and task performance

Best For

Diagnosing reasoning collapse in multi-turn LLM agent RL trainingImproving task performance in planning, math reasoning, web navigation, and code executionSelecting high-quality prompts for agent training iterationsAnalyzing reasoning stability and input dependence in agentic RL

FAQ

What is template collapse?
Template collapse is a failure mode in agentic RL where models rely on fixed input-agnostic reasoning templates that appear diverse but do not actually respond to different inputs, and this collapse is invisible to entropy and all existing metrics.
How does RAGEN-2 diagnose reasoning quality?
RAGEN-2 decomposes reasoning quality into within-input diversity (Entropy) and cross-input distinguishability (Mutual Information), and introduces a family of mutual information proxies for online diagnosis.
What causes template collapse according to the paper?
The paper explains template collapse with a signal-to-noise ratio (SNR) mechanism: low reward variance weakens task gradients, letting regularization terms dominate and erase cross-input reasoning differences.
What solution does RAGEN-2 propose?
RAGEN-2 proposes SNR-Aware Filtering to select high-signal prompts per iteration using reward variance as a lightweight proxy, which consistently improves both input dependence and task performance across planning, math reasoning, web navigation, and code execution.