Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditions

TL;DR

This paper investigates how adverse weather impacts semantic segmentation models in autonomous vehicles, revealing that low-level feature changes cause forgetting and that pre-training with augmentations can mitigate this issue.

Contribution

It identifies the primary cause of catastrophic forgetting as low-level feature changes and demonstrates that pre-training and augmentations promote generalization and reduce forgetting.

Findings

Low-level feature changes drive catastrophic forgetting.

Pre-training and augmentations improve feature generalization.

Generalized features reduce forgetting in domain-incremental learning.

Abstract

Deep neural networks for scene perception in automated vehicles achieve excellent results for the domains they were trained on. However, in real-world conditions, the domain of operation and its underlying data distribution are subject to change. Adverse weather conditions, in particular, can significantly decrease model performance when such data are not available during training.Additionally, when a model is incrementally adapted to a new domain, it suffers from catastrophic forgetting, causing a significant drop in performance on previously observed domains. Despite recent progress in reducing catastrophic forgetting, its causes and effects remain obscure. Therefore, we study how the representations of semantic segmentation models are affected during domain-incremental learning in adverse weather conditions. Our experiments and representational analyses indicate that catastrophic forgetting is primarily caused by changes to low-level features in domain-incremental learning and that learning more general features on the source domain using pre-training and image augmentations leads to efficient feature reuse in subsequent tasks, which drastically reduces catastrophic forgetting. These findings highlight the importance of methods that facilitate generalized features for effective continual learning algorithms.