Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis

TL;DR

This paper introduces a hierarchical attention-based weakly supervised learning method for chest X-ray abnormality localization and diagnosis, improving interpretability and localization accuracy without requiring extensive annotations.

Contribution

The paper proposes a novel hierarchical attention mining framework that combines activation and gradient-based attention with explicit ordinal constraints for weakly supervised localization.

Findings

Significant improvement in localization performance over state-of-the-art methods.

Achieved competitive classification accuracy on large-scale chest X-ray datasets.

Demonstrated the effectiveness of hierarchical attention constraints in medical image analysis.

Abstract

We consider the problem of abnormality localization for clinical applications. While deep learning has driven much recent progress in medical imaging, many clinical challenges are not fully addressed, limiting its broader usage. While recent methods report high diagnostic accuracies, physicians have concerns trusting these algorithm results for diagnostic decision-making purposes because of a general lack of algorithm decision reasoning and interpretability. One potential way to address this problem is to further train these models to localize abnormalities in addition to just classifying them. However, doing this accurately will require a large amount of disease localization annotations by clinical experts, a task that is prohibitively expensive to accomplish for most applications. In this work, we take a step towards addressing these issues by means of a new attention-driven weakly supervised algorithm comprising a hierarchical attention mining framework that unifies activation- and gradient-based visual attention in a holistic manner. Our key algorithmic innovations include the design of explicit ordinal attention constraints, enabling principled model training in a weakly-supervised fashion, while also facilitating the generation of visual-attention-driven model explanations by means of localization cues. On two large-scale chest X-ray datasets (NIH ChestX-ray14 and CheXpert), we demonstrate significant localization performance improvements over the current state of the art while also achieving competitive classification performance. Our code is available on https://github.com/oyxhust/HAM.