Understanding Inter-Concept Relationships in Concept-Based Models

TL;DR

This paper investigates whether concept-based models accurately capture inter-concept relationships, revealing their limitations in stability and robustness, and proposes a new algorithm to leverage these relationships for improved interpretability and downstream task performance.

Contribution

The paper provides an empirical analysis of inter-concept relationship capture in concept-based models and introduces a novel algorithm to enhance their effectiveness by leveraging these relationships.

Findings

State-of-the-art models lack stability and robustness in concept representations.

Current models fail to effectively capture inter-concept relationships.

A new algorithm improves concept intervention accuracy by leveraging inter-concept relationships.

Abstract

Concept-based explainability methods provide insight into deep learning systems by constructing explanations using human-understandable concepts. While the literature on human reasoning demonstrates that we exploit relationships between concepts when solving tasks, it is unclear whether concept-based methods incorporate the rich structure of inter-concept relationships. We analyse the concept representations learnt by concept-based models to understand whether these models correctly capture inter-concept relationships. First, we empirically demonstrate that state-of-the-art concept-based models produce representations that lack stability and robustness, and such methods fail to capture inter-concept relationships. Then, we develop a novel algorithm which leverages inter-concept relationships to improve concept intervention accuracy, demonstrating how correctly capturing inter-concept relationships can improve downstream tasks.