Learning New Concepts, Remembering the Old: Continual Learning for Multimodal Concept Bottleneck Models

TL;DR

This paper introduces CONCIL, a novel continual learning framework for Concept Bottleneck Models that efficiently learns new concepts and classes over time without catastrophic forgetting, suitable for real-time multimodal data.

Contribution

The paper proposes CONCIL, a gradient-free, recursive matrix-based approach for continual learning in CBMs, enabling dynamic concept and class acquisition while preserving prior knowledge.

Findings

CONCIL outperforms traditional CBMs in incremental learning tasks.

It effectively prevents catastrophic forgetting through matrix operations.

Demonstrates high computational efficiency for large-scale multimodal data.

Abstract

Concept Bottleneck Models (CBMs) enhance the interpretability of AI systems, particularly by bridging visual input with human-understandable concepts, effectively acting as a form of multimodal interpretability model. However, existing CBMs typically assume static datasets, which fundamentally limits their adaptability to real-world, continuously evolving multimodal data streams. To address this, we define a novel continual learning task for CBMs: simultaneously handling concept-incremental and class-incremental learning. This task requires models to continuously acquire new concepts (often representing cross-modal attributes) and classes while robustly preserving previously learned knowledge. To tackle this challenging problem, we propose CONceptual Continual Incremental Learning (CONCIL), a novel framework that fundamentally re-imagines concept and decision layer updates as linear regression problems. This reformulation eliminates the need for gradient-based optimization, thereby effectively preventing catastrophic forgetting. Crucially, CONCIL relies solely on recursive matrix operations, rendering it highly computationally efficient and well-suited for real-time and large-scale multimodal data applications. Experimental results compellingly demonstrate that CONCIL achieves "absolute knowledge memory" and significantly surpasses the performance of traditional CBM methods in both concept- and class-incremental settings, thus establishing a new paradigm for continual learning in CBMs, particularly valuable for dynamic multimodal understanding.