Concept Bottleneck with Visual Concept Filtering for Explainable Medical Image Classification

TL;DR

This paper introduces a method to improve interpretability in medical image classification by filtering concepts based on visual relevance, enhancing model performance and concept meaningfulness.

Contribution

It proposes a visual activation score to automatically filter visually irrelevant concepts, improving concept relevance and model interpretability in concept bottleneck models.

Findings

Filtering with visual activation scores boosts classification performance.

Visually relevant concepts are effectively selected, enhancing interpretability.

Method works with unlabeled image data for concept relevance assessment.

Abstract

Interpretability is a crucial factor in building reliable models for various medical applications. Concept Bottleneck Models (CBMs) enable interpretable image classification by utilizing human-understandable concepts as intermediate targets. Unlike conventional methods that require extensive human labor to construct the concept set, recent works leveraging Large Language Models (LLMs) for generating concepts made automatic concept generation possible. However, those methods do not consider whether a concept is visually relevant or not, which is an important factor in computing meaningful concept scores. Therefore, we propose a visual activation score that measures whether the concept contains visual cues or not, which can be easily computed with unlabeled image data. Computed visual activation scores are then used to filter out the less visible concepts, thus resulting in a final concept set with visually meaningful concepts. Our experimental results show that adopting the proposed visual activation score for concept filtering consistently boosts performance compared to the baseline. Moreover, qualitative analyses also validate that visually relevant concepts are successfully selected with the visual activation score.