Medical Reasoning with Large Language Models: A Survey and MR-Bench

TL;DR

This paper reviews medical reasoning with large language models, introduces MR-Bench for real-world clinical evaluation, and highlights gaps between current performance and clinical needs.

Contribution

It provides a comprehensive survey of medical reasoning methods, organizes them into seven routes, and introduces MR-Bench for authentic clinical assessment.

Findings

Models perform well on exam-style tasks but poorly on real clinical data.

MR-Bench reveals a significant gap between exam performance and clinical accuracy.

Unified evaluation exposes the need for improved reasoning in clinical settings.

Abstract

Large language models (LLMs) have achieved strong performance on medical exam-style tasks, motivating growing interest in their deployment in real-world clinical settings. However, clinical decision-making is inherently safety-critical, context-dependent, and conducted under evolving evidence. In such situations, reliable LLM performance depends not on factual recall alone, but on robust medical reasoning. In this work, we present a comprehensive review of medical reasoning with LLMs. Grounded in cognitive theories of clinical reasoning, we conceptualize medical reasoning as an iterative process of abduction, deduction, and induction, and organize existing methods into seven major technical routes spanning training-based and training-free approaches. We further conduct a unified cross-benchmark evaluation of representative medical reasoning models under a consistent experimental setting, enabling a more systematic and comparable assessment of the empirical impact of existing methods. To better assess clinically grounded reasoning, we introduce MR-Bench, a benchmark derived from real-world hospital data. Evaluations on MR-Bench expose a pronounced gap between exam-level performance and accuracy on authentic clinical decision tasks. Overall, this survey provides a unified view of existing medical reasoning methods, benchmarks, and evaluation practices, and highlights key gaps between current model performance and the requirements of real-world clinical reasoning.