DCBM: Data-Efficient Visual Concept Bottleneck Models

TL;DR

DCBMs improve the interpretability and data efficiency of concept bottleneck models by using image region detection instead of large language models, enabling better performance in data-sparse and out-of-distribution scenarios.

Contribution

Introduction of Data-efficient CBMs that utilize image region detection for concept generation, reducing reliance on large datasets and enhancing adaptability.

Findings

Effective concept localization demonstrated with Grad-CAM.

Improved performance in fine-grained and out-of-distribution tasks.

Reduced data requirements for concept generation.

Abstract

Concept Bottleneck Models (CBMs) enhance the interpretability of neural networks by basing predictions on human-understandable concepts. However, current CBMs typically rely on concept sets extracted from large language models or extensive image corpora, limiting their effectiveness in data-sparse scenarios. We propose Data-efficient CBMs (DCBMs), which reduce the need for large sample sizes during concept generation while preserving interpretability. DCBMs define concepts as image regions detected by segmentation or detection foundation models, allowing each image to generate multiple concepts across different granularities. This removes reliance on textual descriptions and large-scale pre-training, making DCBMs applicable for fine-grained classification and out-of-distribution tasks. Attribution analysis using Grad-CAM demonstrates that DCBMs deliver visual concepts that can be localized in test images. By leveraging dataset-specific concepts instead of predefined ones, DCBMs enhance adaptability to new domains.