Reading Radiology Imaging Like The Radiologist

TL;DR

This paper introduces a disease-oriented retrieval framework and a factual consistency captioning model to improve the accuracy and relevance of automated radiology report generation, addressing subtle image differences and data biases.

Contribution

It proposes a novel retrieval-based approach combined with a factual consistency generator to enhance report accuracy in medical imaging analysis.

Findings

Improved report accuracy with disease-oriented retrieval.

Enhanced factual consistency in generated reports.

Better handling of subtle image differences and data biases.

Abstract

Automated radiology report generation aims to generate radiology reports that contain rich, fine-grained descriptions of radiology imaging. Compared with image captioning in the natural image domain, medical images are very similar to each other, with only minor differences in the occurrence of diseases. Given the importance of these minor differences in the radiology report, it is crucial to encourage the model to focus more on the subtle regions of disease occurrence. Secondly, the problem of visual and textual data biases is serious. Not only do normal cases make up the majority of the dataset, but sentences describing areas with pathological changes also constitute only a small part of the paragraph. Lastly, generating medical image reports involves the challenge of long text generation, which requires more expertise and empirical training in medical knowledge. As a result, the difficulty of generating such reports is increased. To address these challenges, we propose a disease-oriented retrieval framework that utilizes similar reports as prior knowledge references. We design a factual consistency captioning generator to generate more accurate and factually consistent disease descriptions. Our framework can find most similar reports for a given disease from the CXR database by retrieving a disease-oriented mask consisting of the position and morphological characteristics. By referencing the disease-oriented similar report and the visual features, the factual consistency model can generate a more accurate radiology report.