MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment

TL;DR

This paper introduces MICA, a multi-level image-concept alignment framework for explainable skin lesion diagnosis that improves interpretability and performance by semantically aligning images and concepts at multiple levels.

Contribution

It proposes a novel multi-modal framework that aligns medical images and concepts across multiple semantic levels, enhancing interpretability and diagnostic accuracy.

Findings

Achieves high performance and label efficiency in skin lesion diagnosis

Provides both textual and visual explanations for model decisions

Maintains interpretability while improving diagnostic accuracy

Abstract

Black-box deep learning approaches have showcased significant potential in the realm of medical image analysis. However, the stringent trustworthiness requirements intrinsic to the medical field have catalyzed research into the utilization of Explainable Artificial Intelligence (XAI), with a particular focus on concept-based methods. Existing concept-based methods predominantly apply concept annotations from a single perspective (e.g., global level), neglecting the nuanced semantic relationships between sub-regions and concepts embedded within medical images. This leads to underutilization of the valuable medical information and may cause models to fall short in harmoniously balancing interpretability and performance when employing inherently interpretable architectures such as Concept Bottlenecks. To mitigate these shortcomings, we propose a multi-modal explainable disease diagnosis framework that meticulously aligns medical images and clinical-related concepts semantically at multiple strata, encompassing the image level, token level, and concept level. Moreover, our method allows for model intervention and offers both textual and visual explanations in terms of human-interpretable concepts. Experimental results on three skin image datasets demonstrate that our method, while preserving model interpretability, attains high performance and label efficiency for concept detection and disease diagnosis.