Learning Concept Bottleneck Models from Mechanistic Explanations

TL;DR

This paper introduces Mechanistic CBMs, a new approach that constructs interpretable concept bottlenecks directly from a black-box model’s learned concepts, improving interpretability and predictive power.

Contribution

We propose a novel CBM pipeline that extracts concepts from a black-box model using autoencoders and LLMs, enhancing interpretability and performance over prior methods.

Findings

M-CBMs outperform prior CBMs at matched sparsity

Improved concept prediction accuracy

Provides concise, interpretable explanations

Abstract

Concept Bottleneck Models (CBMs) aim for ante-hoc interpretability by learning a bottleneck layer that predicts interpretable concepts before the decision. State-of-the-art approaches typically select which concepts to learn via human specification, open knowledge graphs, prompting an LLM, or using general CLIP concepts. However, concepts defined a-priori may not have sufficient predictive power for the task or even be learnable from the available data. As a result, these CBMs often significantly trail their black-box counterpart when controlling for information leakage. To address this, we introduce a novel CBM pipeline named Mechanistic CBM (M-CBM), which builds the bottleneck directly from a black-box model's own learned concepts. These concepts are extracted via Sparse Autoencoders (SAEs) and subsequently named and annotated on a selected subset of images using a Multimodal LLM. For fair comparison and leakage control, we also introduce the Number of Contributing Concepts (NCC), a decision-level sparsity metric that extends the recently proposed NEC metric. Across diverse datasets, we show that M-CBMs consistently surpass prior CBMs at matched sparsity, while improving concept predictions and providing concise explanations. Our code is available at https://github.com/Antonio-Dee/M-CBM.