LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation

TL;DR

LEAD is a novel decoding approach that integrates expert features at each layer of large vision language models to improve factual accuracy and reduce hallucinations in radiology report generation.

Contribution

The paper introduces Layer-wise Expert-aligned Decoding (LEAD), a new method that dynamically incorporates expert features during decoding to enhance factual consistency.

Findings

LEAD reduces hallucinations in generated reports.

LEAD improves clinical accuracy metrics.

LEAD maintains high report fluency and quality.

Abstract

Radiology Report Generation (RRG) aims to produce accurate and coherent diagnostics from medical images. Although large vision language models (LVLM) improve report fluency and accuracy, they exhibit hallucinations, generating plausible yet image-ungrounded pathological details. Existing methods primarily rely on external knowledge guidance to facilitate the alignment between generated text and visual information. However, these approaches often ignore the inherent decoding priors and vision-language alignment biases in pretrained models and lack robustness due to reliance on constructed guidance. In this paper, we propose Layer-wise Expert-aligned Decoding (LEAD), a novel method to inherently modify the LVLM decoding trajectory. A multiple experts module is designed for extracting distinct pathological features which are integrated into each decoder layer via a gating mechanism. This layer-wise architecture enables the LLM to consult expert features at every inference step via a learned gating function, thereby dynamically rectifying decoding biases and steering the generation toward factual consistency. Experiments conducted on multiple public datasets demonstrate that the LEAD method yields effective improvements in clinical accuracy metrics and mitigates hallucinations while preserving high generation quality.