RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation

TL;DR

This paper enhances continual learning in semantic segmentation by using adversarial web data retrieval and adaptive strategies to mitigate catastrophic forgetting across multiple incremental steps.

Contribution

It introduces adversarial and adaptive threshold techniques to select web data that closely matches previous training data, improving continual learning performance.

Findings

Achieves significant performance improvements in multi-step incremental scenarios.

Effectively retrieves relevant old class examples from web data.

Enhances pseudo-labeling for better web data annotation.

Abstract

Catastrophic forgetting of previous knowledge is a critical issue in continual learning typically handled through various regularization strategies. However, existing methods struggle especially when several incremental steps are performed. In this paper, we extend our previous approach (RECALL) and tackle forgetting by exploiting unsupervised web-crawled data to retrieve examples of old classes from online databases. In contrast to the original methodology, which did not incorporate an assessment of web-based data, the present work proposes two advanced techniques: an adversarial approach and an adaptive threshold strategy. These methods are utilized to meticulously choose samples from web data that exhibit strong statistical congruence with the no longer available training data. Furthermore, we improved the pseudo-labeling scheme to achieve a more accurate labeling of web data that also considers classes being learned in the current step. Experimental results show that this enhanced approach achieves remarkable results, particularly when the incremental scenario spans multiple steps.