Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation

TL;DR

This paper introduces a novel attribution-aware weight transfer method for class-incremental semantic segmentation that improves learning efficiency and reduces forgetting by better initializing classifiers for new classes using gradient-based attribution.

Contribution

It proposes a new classifier initialization technique using attribution to address background shift and enhance incremental learning in semantic segmentation.

Findings

Significant mIoU improvements over state-of-the-art methods.

Effective mitigation of catastrophic forgetting.

Accelerated learning of new classes.

Abstract

In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. Although recent works focused on these issues, existing classifier initialization methods do not address the background shift problem and assign the same initialization weights to both background and new foreground class classifiers. We propose to address the background shift with a novel classifier initialization method which employs gradient-based attribution to identify the most relevant weights for new classes from the classifier's weights for the previous background and transfers these weights to the new classifier. This warm-start weight initialization provides a general solution applicable to several CISS methods. Furthermore, it accelerates learning of new classes while mitigating forgetting. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.