Completely Weakly Supervised Class-Incremental Learning for Semantic Segmentation

TL;DR

This paper introduces a novel completely weakly supervised class-incremental learning method for semantic segmentation, using only image-level labels, and demonstrates its effectiveness across multiple datasets and settings.

Contribution

It is the first to propose a completely weakly-supervised approach for class-incremental semantic segmentation, combining pseudo-labels from localizers and foundation models, and using exemplar-guided data augmentation.

Findings

Outperforms partially weakly supervised methods on VOC datasets.

Achieves competitive accuracy on COCO-to-VOC setting.

Effective in disjoint and overlap scenarios.

Abstract

This work addresses the task of completely weakly supervised class-incremental learning for semantic segmentation to learn segmentation for both base and additional novel classes using only image-level labels. While class-incremental semantic segmentation (CISS) is crucial for handling diverse and newly emerging objects in the real world, traditional CISS methods require expensive pixel-level annotations for training. To overcome this limitation, partially weakly-supervised approaches have recently been proposed. However, to the best of our knowledge, this is the first work to introduce a completely weakly-supervised method for CISS. To achieve this, we propose to generate robust pseudo-labels by combining pseudo-labels from a localizer and a sequence of foundation models based on their uncertainty. Moreover, to mitigate catastrophic forgetting, we introduce an exemplar-guided data augmentation method that generates diverse images containing both previous and novel classes with guidance. Finally, we conduct experiments in three common experimental settings: 15-5 VOC, 10-10 VOC, and COCO-to-VOC, and in two scenarios: disjoint and overlap. The experimental results demonstrate that our completely weakly supervised method outperforms even partially weakly supervised methods in the 15-5 VOC and 10-10 VOC settings while achieving competitive accuracy in the COCO-to-VOC setting.