Matryoshka Concept Bottleneck Models

TL;DR

This paper introduces the Matryoshka Concept Bottleneck Model (MCBM), a hierarchical approach that reduces intervention costs and improves interpretability in concept-based deep learning models.

Contribution

The paper proposes MCBM, a unified hierarchical architecture that enables adaptive concept utilization and reduces intervention costs from linear to logarithmic scale.

Findings

MCBM matches the performance of separate models.

Reduces expected intervention costs to O(log K).

Enables dynamic and efficient expert interaction.

Abstract

Concept Bottleneck Models (CBMs) have emerged as a prominent paradigm for interpretable deep learning, learning by grounding predictions in human-understandable concepts. However, their practical deployment is hindered by the high cost of test-time intervention, as correcting model errors typically requires human experts to manually inspect and verify a large set of predicted concepts. Existing approaches suffer from a fundamental structural limitation: they either adopt a single static concept set, forcing experts to exhaustively annotate concepts and incurring prohibitive intervention costs, or train multiple models tailored to different concept budgets, resulting in substantial computational and maintenance overhead. To address this challenge, we propose the Matryoshka Concept Bottleneck Model (MCBM), a unified architecture that enables adaptive concept utilization within a single model. Inspired by Matryoshka Representation Learning, MCBM organizes concepts into a nested hierarchy based on maximum relevance and minimum redundancy, allowing inference at multiple levels of conceptual granularity without retraining. Theoretically, we show that MCBM reduces the expected intervention costs from linear to logarithmic order, $O(\log K)$, while guaranteeing monotonic performance improvement. Empirically, extensive experiments demonstrate that MCBM matches the performance of independently trained models while enabling dynamic and efficient expert interaction.