Looking in the Right place for Anomalies: Explainable AI through Automatic Location Learning

TL;DR

This paper introduces a novel explainable AI approach for medical image anomaly detection that guarantees the network focuses on the expected anomaly location by automatically learning from textual reports and guiding the model's attention.

Contribution

It presents a hybrid model combining Bi-LSTM and DenseNet-121 to learn expected anomaly locations from reports, improving explainability and accuracy in medical imaging.

Findings

Method effectively overlaps anomalies with expected locations.

Improves interpretability over traditional heat map methods.

Validated on a large chest X-ray dataset.

Abstract

Deep learning has now become the de facto approach to the recognition of anomalies in medical imaging. Their 'black box' way of classifying medical images into anomaly labels poses problems for their acceptance, particularly with clinicians. Current explainable AI methods offer justifications through visualizations such as heat maps but cannot guarantee that the network is focusing on the relevant image region fully containing the anomaly. In this paper, we develop an approach to explainable AI in which the anomaly is assured to be overlapping the expected location when present. This is made possible by automatically extracting location-specific labels from textual reports and learning the association of expected locations to labels using a hybrid combination of Bi-Directional Long Short-Term Memory Recurrent Neural Networks (Bi-LSTM) and DenseNet-121. Use of this expected location to bias the subsequent attention-guided inference network based on ResNet101 results in the isolation of the anomaly at the expected location when present. The method is evaluated on a large chest X-ray dataset.