Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization

TL;DR

This paper evaluates hallucination detection in medical text summarization, showing general detectors struggle with clinical data, and introduces fact-based, explainable methods that generalize well to real-world clinical hallucinations.

Contribution

The study constructs specialized datasets and develops fact-based hallucination detection methods that improve explainability and generalization in clinical summarization.

Findings

General-domain detectors underperform on clinical hallucinations.

Fact-controlled datasets reveal limitations of existing detectors.

LLM-based detectors trained on fact-controlled data generalize to real clinical hallucinations.

Abstract

Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems.