Anatomy-Guided Radiology Report Generation with Pathology-Aware Regional Prompts

TL;DR

This paper introduces a novel radiology report generation method that uses pathology-aware regional prompts and specialized detectors to improve clinical accuracy and relevance of generated reports.

Contribution

It proposes an anatomy-guided approach with regional prompts and detectors to better incorporate pathological information, surpassing previous fixed-feature methods.

Findings

Outperforms state-of-the-art on natural language metrics

Achieves higher clinical efficacy scores

Expert evaluations confirm improved report quality

Abstract

Radiology reporting generative AI holds significant potential to alleviate clinical workloads and streamline medical care. However, achieving high clinical accuracy is challenging, as radiological images often feature subtle lesions and intricate structures. Existing systems often fall short, largely due to their reliance on fixed size, patch-level image features and insufficient incorporation of pathological information. This can result in the neglect of such subtle patterns and inconsistent descriptions of crucial pathologies. To address these challenges, we propose an innovative approach that leverages pathology-aware regional prompts to explicitly integrate anatomical and pathological information of various scales, significantly enhancing the precision and clinical relevance of generated reports. We develop an anatomical region detector that extracts features from distinct anatomical areas, coupled with a novel multi-label lesion detector that identifies global pathologies. Our approach emulates the diagnostic process of radiologists, producing clinically accurate reports with comprehensive diagnostic capabilities. Experimental results show that our model outperforms previous state-of-the-art methods on most natural language generation and clinical efficacy metrics, with formal expert evaluations affirming its potential to enhance radiology practice.