Mixture of Concept Bottleneck Experts

TL;DR

This paper introduces Mixture of Concept Bottleneck Experts, a flexible framework that enhances interpretability and accuracy in concept bottleneck models by combining multiple experts with different functional forms.

Contribution

It proposes a novel framework that generalizes CBMs by incorporating multiple experts and functional forms, including symbolic regression, to improve adaptability and performance.

Findings

Varying mixture size improves accuracy-interpretability balance.

Linear and symbolic expert models outperform single-expert CBMs.

Framework adapts to diverse user and task requirements.

Abstract

Concept Bottleneck Models (CBMs) promote interpretability by grounding predictions in human-understandable concepts. However, existing CBMs typically fix their task predictor to a single linear or Boolean expression, limiting both predictive accuracy and adaptability to diverse user needs. We propose Mixture of Concept Bottleneck Experts (M-CBEs), a framework that generalizes existing CBMs along two dimensions: the number of experts and the functional form of each expert, exposing an underexplored region of the design space. We investigate this region by instantiating two novel models: Linear M-CBE, which learns a finite set of linear expressions, and Symbolic M-CBE, which leverages symbolic regression to discover expert functions from data under user-specified operator vocabularies. Empirical evaluation demonstrates that varying the mixture size and functional form provides a robust framework for navigating the accuracy-interpretability trade-off, adapting to different user and task needs.