Cognitive bias in LLM reasoning compromises interpretation of clinical oncology notes

TL;DR

This study reveals that large language models often make reasoning errors in clinical oncology notes, which can lead to unsafe recommendations, highlighting the need for improved evaluation frameworks before clinical use.

Contribution

The paper introduces a hierarchical taxonomy of reasoning errors in LLMs applied to oncology, linking computational failures to cognitive biases and validating it across multiple cancer types.

Findings

Reasoning errors occurred in 23% of interpretations.

Confirmation and anchoring biases were most common.

Errors were associated with guideline-discordant and harmful recommendations.

Abstract

Despite high performance on clinical benchmarks, large language models may reach correct conclusions through faulty reasoning, a failure mode with safety implications for oncology decision support that is not captured by accuracy-based evaluation. In this two-cohort retrospective study, we developed a hierarchical taxonomy of reasoning errors from GPT-4 chain-of-thought responses to real oncology notes and tested its clinical relevance. Using breast and pancreatic cancer notes from the CORAL dataset, we annotated 600 reasoning traces to define a three-tier taxonomy mapping computational failures to cognitive bias frameworks. We validated the taxonomy on 822 responses from prostate cancer consult notes spanning localized through metastatic disease, simulating extraction, analysis, and clinical recommendation tasks. Reasoning errors occurred in 23 percent of interpretations and dominated overall errors, with confirmation bias and anchoring bias most common. Reasoning failures were associated with guideline-discordant and potentially harmful recommendations, particularly in advanced disease management. Automated evaluators using state-of-the-art language models detected error presence but could not reliably classify subtypes. These findings show that large language models may provide fluent but clinically unsafe recommendations when reasoning is flawed. The taxonomy provides a generalizable framework for evaluating and improving reasoning fidelity before clinical deployment.