Medical Reasoning with Large Language Models: A Systematic Review and Evaluation (April 2026)

Free

Comprehensive review of medical reasoning methods + MR-Bench (real-world hospital data); reveals large gap between exam-level performance and authentic clinical decision-making

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About Medical Reasoning with Large Language Models: A Systematic Review and Evaluation (April 2026)

This paper presents a comprehensive survey of medical reasoning with large language models (LLMs), grounded in cognitive theories of clinical reasoning and organizing existing methods into seven major technical routes spanning training-based and training-free approaches. The authors introduce MR-Bench, a benchmark derived from real-world hospital data designed to assess clinically grounded reasoning. Evaluations on MR-Bench reveal a pronounced gap between LLMs' strong performance on medical exam-style tasks and their accuracy on authentic clinical decision-making tasks, highlighting the need for robust reasoning beyond factual recall.

Key Features

Comprehensive review of medical reasoning methods with LLMs

MR-Bench benchmark derived from real-world hospital data

Unified cross-benchmark evaluation under consistent experimental setting

Organizes methods into seven technical routes (training-based and training-free)

Grounded in cognitive theories of abduction, deduction, and induction

Exposes pronounced gap between exam-level performance and authentic clinical decision tasks

Pros & Cons

Pros

Provides a systematic and unified overview of existing medical reasoning methods
Introduces a benchmark built from actual clinical data, not synthetic exams
Reveals critical gap that is important for safe deployment in healthcare
Grounds analysis in established cognitive theories of clinical reasoning
Open access and freely available on arXiv

Cons

Paper is a survey and benchmark, not a deployable clinical tool
Benchmark may not cover all medical specialties or settings
Findings may become outdated as models evolve rapidly

Best For

Evaluating LLMs for real-world clinical decision-makingGuiding future research on medical reasoning in AIBenchmarking medical LLMs against hospital-derived dataUnderstanding limitations of current LLMs in safety-critical settings

FAQ

What is MR-Bench?

MR-Bench is a benchmark introduced in the paper, derived from real-world hospital data, designed to assess LLMs' performance on authentic clinical decision tasks.

What medical reasoning methods are covered in the survey?

The survey covers seven major technical routes, including both training-based and training-free approaches, grounded in cognitive theories of abduction, deduction, and induction.

Does this paper provide a working system or model?

No, the paper is a comprehensive review and evaluation of existing methods, along with a new benchmark. It does not release a new LLM or deployable system.