Reasoning about uncertainty: Do reasoning models know when they don't know?

Analysis

Why This Paper Matters

Reasoning models, which generate step-by-step chains of thought, are increasingly deployed in critical applications where incorrect answers can have severe consequences. However, their ability to recognize their own uncertainty has been largely unexplored. Prior uncertainty quantification (UQ) research has focused on non-reasoning models, leaving a gap in understanding how reasoning models express and handle uncertainty. This paper addresses that gap by evaluating reasoning models without fine-tuning, using self-verbalized UQ methods—where the model is prompted to state its confidence. This is significant because it tests the models' intrinsic ability to introspect, which is essential for building trustworthy AI systems.

The findings have immediate practical implications. If reasoning models are overconfident, users may over-rely on their outputs, leading to errors in decision-making. Conversely, if they are well-calibrated, they can be used more safely. By focusing on zero-shot self-verbalization, the paper provides a cost-effective way to assess uncertainty without additional training, which is particularly relevant for large models where fine-tuning is expensive.

Technical Contributions

• Zero-shot self-verbalized UQ: The paper applies methods that prompt the model to verbalize its confidence (e.g., "How confident are you?") without any fine-tuning, preserving the model's original capabilities.
• Comparison with non-reasoning baselines: The study compares reasoning models with standard LLMs to isolate the effect of reasoning on uncertainty calibration.
• Benchmark evaluation: Uses established uncertainty benchmarks to measure calibration, such as expected calibration error (ECE) and selective prediction metrics.
• Failure mode analysis: Identifies specific conditions (e.g., complex multi-step reasoning) where self-verbalized UQ fails, providing insights into the limitations of introspection.

Results

While the abstract does not provide specific numbers, the paper reports that reasoning models show varying calibration performance. Self-verbalized UQ methods are effective in some cases but exhibit overconfidence in others, particularly when the reasoning chain is long or the problem is ambiguous. The comparison with non-reasoning models suggests that reasoning does not automatically improve uncertainty awareness; in fact, it may sometimes exacerbate overconfidence due to the model's commitment to its generated chain of thought.

Significance

This research is a stepping stone for improving uncertainty in reasoning models. It highlights that self-verbalization is a promising but imperfect tool, and that more robust methods are needed. The findings could influence how reasoning models are deployed in high-stakes domains like healthcare and finance, where calibrated uncertainty is critical. Moreover, it opens avenues for future work on fine-tuning for calibration, integrating external confidence signals, and developing hybrid UQ methods that combine verbalized confidence with statistical approaches. As reasoning models become more prevalent, understanding their epistemic limits will be key to ensuring safe and reliable AI.