Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation

TL;DR

This paper introduces a novel framework called CIPL that improves multi-label disease diagnosis and interpretation in medical images by disentangling multiple diseases and leveraging intra-image consistency, achieving state-of-the-art results.

Contribution

The paper proposes a new Cross- and Intra-image Prototypical Learning (CIPL) framework with a two-level regularization strategy for better multi-label disease diagnosis and interpretability.

Findings

CIPL achieves state-of-the-art accuracy on thoracic radiography and fundus image benchmarks.

CIPL provides superior weakly-supervised disease localization compared to existing methods.

The framework effectively disentangles multiple diseases within images for improved interpretability.

Abstract

Recent advances in prototypical learning have shown remarkable potential to provide useful decision interpretations associating activation maps and predictions with class-specific training prototypes. Such prototypical learning has been well-studied for various single-label diseases, but for quite relevant and more challenging multi-label diagnosis, where multiple diseases are often concurrent within an image, existing prototypical learning models struggle to obtain meaningful activation maps and effective class prototypes due to the entanglement of the multiple diseases. In this paper, we present a novel Cross- and Intra-image Prototypical Learning (CIPL) framework, for accurate multi-label disease diagnosis and interpretation from medical images. CIPL takes advantage of common cross-image semantics to disentangle the multiple diseases when learning the prototypes, allowing a comprehensive understanding of complicated pathological lesions. Furthermore, we propose a new two-level alignment-based regularisation strategy that effectively leverages consistent intra-image information to enhance interpretation robustness and predictive performance. Extensive experiments show that our CIPL attains the state-of-the-art (SOTA) classification accuracy in two public multi-label benchmarks of disease diagnosis: thoracic radiography and fundus images. Quantitative interpretability results show that CIPL also has superiority in weakly-supervised thoracic disease localisation over other leading saliency- and prototype-based explanation methods.