Modeling Clinical Uncertainty in Radiology Reports: from Explicit Uncertainty Markers to Implicit Reasoning Pathways

TL;DR

This paper introduces a framework to quantify explicit and implicit uncertainty in radiology reports, enhancing automated analysis by modeling uncertainty markers and reasoning pathways, and releases an enriched benchmark dataset.

Contribution

It presents a novel two-part framework for modeling both explicit and implicit uncertainty in radiology reports, including an expert-validated ranking system and diagnostic pathway expansion.

Findings

Quantified explicit uncertainty using a reference ranking of hedging phrases.

Modeled implicit uncertainty through systematic expansion of diagnostic pathways.

Released an uncertainty-aware, expanded radiology report benchmark dataset.

Abstract

Radiology reports are invaluable for clinical decision-making and hold great potential for automated analysis when structured into machine-readable formats. These reports often contain uncertainty, which we categorize into two distinct types: (i) Explicit uncertainty reflects doubt about the presence or absence of findings, conveyed through hedging phrases. These vary in meaning depending on the context, making rule-based systems insufficient to quantify the level of uncertainty for specific findings; (ii) Implicit uncertainty arises when radiologists omit parts of their reasoning, recording only key findings or diagnoses. Here, it is often unclear whether omitted findings are truly absent or simply unmentioned for brevity. We address these challenges with a two-part framework. We quantify explicit uncertainty by creating an expert-validated, LLM-based reference ranking of common hedging phrases, and mapping each finding to a probability value based on this reference. In addition, we model implicit uncertainty through an expansion framework that systematically adds characteristic sub-findings derived from expert-defined diagnostic pathways for 14 common diagnoses. Using these methods, we release Lunguage++, an expanded, uncertainty-aware version of the Lunguage benchmark of fine-grained structured radiology reports. This enriched resource enables uncertainty-aware image classification, faithful diagnostic reasoning, and new investigations into the clinical impact of diagnostic uncertainty.