# Learning Yourself: Class-Incremental Semantic Segmentation with Language-Inspired Bootstrapped Disentanglement

## TL;DR

This paper introduces a novel framework called LBD that leverages pre-trained visual-language models to improve class-incremental semantic segmentation by disentangling features guided by language semantics, achieving state-of-the-art results.

## Contribution

The paper proposes a language-inspired bootstrapped disentanglement framework that uses CLIP's semantic prior to address catastrophic semantic entanglement in incremental segmentation tasks.

## Key findings

- Achieves state-of-the-art performance on Pascal VOC and ADE20k datasets.
- Effectively disentangles features using language-guided prototypes.
- Improves multi-step incremental segmentation accuracy.

## Abstract

Class-Incremental Semantic Segmentation (CISS) requires continuous learning of newly introduced classes while retaining knowledge of past classes. By abstracting mainstream methods into two stages (visual feature extraction and prototype-feature matching), we identify a more fundamental challenge termed catastrophic semantic entanglement. This phenomenon involves Prototype-Feature Entanglement caused by semantic misalignment during the incremental process, and Background-Increment Entanglement due to dynamic data evolution. Existing techniques, which rely on visual feature learning without sufficient cues to distinguish targets, introduce significant noise and errors. To address these issues, we introduce a Language-inspired Bootstrapped Disentanglement framework (LBD). We leverage the prior class semantics of pre-trained visual-language models (e.g., CLIP) to guide the model in autonomously disentangling features through Language-guided Prototypical Disentanglement and Manifold Mutual Background Disentanglement. The former guides the disentangling of new prototypes by treating hand-crafted text features as topological templates, while the latter employs multiple learnable prototypes and mask-pooling-based supervision for background-incremental class disentanglement. By incorporating soft prompt tuning and encoder adaptation modifications, we further bridge the capability gap of CLIP between dense and sparse tasks, achieving state-of-the-art performance on both Pascal VOC and ADE20k, particularly in multi-step scenarios.