AWF: Adaptive Weight Fusion for Enhanced Class Incremental Semantic Segmentation

TL;DR

This paper introduces Adaptive Weight Fusion (AWF), a novel method for class incremental semantic segmentation that adaptively fuses model weights to better retain old knowledge while learning new classes, outperforming previous methods.

Contribution

The paper proposes AWF, an enhanced weight fusion approach with an alternating training strategy for the fusion parameter, improving knowledge retention and learning in CISS tasks.

Findings

AWF outperforms existing methods on benchmark CISS tasks.

Adaptive fusion leads to better balance between old and new knowledge.

Experiment code will be released on Github.

Abstract

Class Incremental Semantic Segmentation (CISS) aims to mitigate catastrophic forgetting by maintaining a balance between previously learned and newly introduced knowledge. Existing methods, primarily based on regularization techniques like knowledge distillation, help preserve old knowledge but often face challenges in effectively integrating new knowledge, resulting in limited overall improvement. Endpoints Weight Fusion (EWF) method, while simple, effectively addresses some of these limitations by dynamically fusing the model weights from previous steps with those from the current step, using a fusion parameter alpha determined by the relative number of previously known classes and newly introduced classes. However, the simplicity of the alpha calculation may limit its ability to fully capture the complexities of different task scenarios, potentially leading to suboptimal fusion outcomes. In this paper, we propose an enhanced approach called Adaptive Weight Fusion (AWF), which introduces an alternating training strategy for the fusion parameter, allowing for more flexible and adaptive weight integration. AWF achieves superior performance by better balancing the retention of old knowledge with the learning of new classes, significantly improving results on benchmark CISS tasks compared to the original EWF. And our experiment code will be released on Github.