Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting

TL;DR

This paper introduces finding-aware anatomical tokens derived from localized anatomical structures in chest X-ray images, significantly improving the accuracy of automated radiology reports by integrating these tokens into the reporting pipeline.

Contribution

It proposes a novel adaptation of Faster R-CNN to generate anatomically and finding-aware tokens, enhancing report quality over previous image-level feature methods.

Findings

Achieved state-of-the-art performance on MIMIC-CXR dataset.

Generated reports with improved clinical accuracy.

Demonstrated the effectiveness of anatomical tokens in radiology reporting.

Abstract

The task of radiology reporting comprises describing and interpreting the medical findings in radiographic images, including description of their location and appearance. Automated approaches to radiology reporting require the image to be encoded into a suitable token representation for input to the language model. Previous methods commonly use convolutional neural networks to encode an image into a series of image-level feature map representations. However, the generated reports often exhibit realistic style but imperfect accuracy. Inspired by recent works for image captioning in the general domain in which each visual token corresponds to an object detected in an image, we investigate whether using local tokens corresponding to anatomical structures can improve the quality of the generated reports. We introduce a novel adaptation of Faster R-CNN in which finding detection is performed for the candidate bounding boxes extracted during anatomical structure localisation. We use the resulting bounding box feature representations as our set of finding-aware anatomical tokens. This encourages the extracted anatomical tokens to be informative about the findings they contain (required for the final task of radiology reporting). Evaluating on the MIMIC-CXR dataset of chest X-Ray images, we show that task-aware anatomical tokens give state-of-the-art performance when integrated into an automated reporting pipeline, yielding generated reports with improved clinical accuracy.