Editable Concept Bottleneck Models

TL;DR

This paper introduces Editable Concept Bottleneck Models (ECBMs) that enable efficient modification of trained CBMs by removing or inserting data and concepts without retraining, using influence functions.

Contribution

The paper proposes ECBMs with closed-form influence function approximations for data and concept editing, addressing scalability and practicality issues in CBMs.

Findings

ECBMs support concept and data removal at multiple levels.

ECBMs achieve efficient editing without retraining.

Experimental results confirm the effectiveness of ECBMs.

Abstract

Concept Bottleneck Models (CBMs) have garnered much attention for their ability to elucidate the prediction process through a humanunderstandable concept layer. However, most previous studies focused on cases where the data, including concepts, are clean. In many scenarios, we often need to remove/insert some training data or new concepts from trained CBMs for reasons such as privacy concerns, data mislabelling, spurious concepts, and concept annotation errors. Thus, deriving efficient editable CBMs without retraining from scratch remains a challenge, particularly in large-scale applications. To address these challenges, we propose Editable Concept Bottleneck Models (ECBMs). Specifically, ECBMs support three different levels of data removal: concept-label-level, concept-level, and data-level. ECBMs enjoy mathematically rigorous closed-form approximations derived from influence functions that obviate the need for retraining. Experimental results demonstrate the efficiency and adaptability of our ECBMs, affirming their practical value in CBMs.