Multi-Domain Incremental Learning for Semantic Segmentation

TL;DR

This paper introduces a dynamic architecture and optimization strategy for multi-domain incremental learning in semantic segmentation, enabling models to learn new geographical domains sequentially without forgetting previous ones.

Contribution

It proposes a novel dynamic architecture with shared and domain-specific parameters, along with an optimization method to balance stability and plasticity in incremental learning.

Findings

Effective in retaining old domain knowledge while learning new domains

Demonstrated on datasets from Germany, US, and India

Outperforms baseline methods in domain incremental settings

Abstract

Recent efforts in multi-domain learning for semantic segmentation attempt to learn multiple geographical datasets in a universal, joint model. A simple fine-tuning experiment performed sequentially on three popular road scene segmentation datasets demonstrates that existing segmentation frameworks fail at incrementally learning on a series of visually disparate geographical domains. When learning a new domain, the model catastrophically forgets previously learned knowledge. In this work, we pose the problem of multi-domain incremental learning for semantic segmentation. Given a model trained on a particular geographical domain, the goal is to (i) incrementally learn a new geographical domain, (ii) while retaining performance on the old domain, (iii) given that the previous domain's dataset is not accessible. We propose a dynamic architecture that assigns universally shared, domain-invariant parameters to capture homogeneous semantic features present in all domains, while dedicated domain-specific parameters learn the statistics of each domain. Our novel optimization strategy helps achieve a good balance between retention of old knowledge (stability) and acquiring new knowledge (plasticity). We demonstrate the effectiveness of our proposed solution on domain incremental settings pertaining to real-world driving scenes from roads of Germany (Cityscapes), the United States (BDD100k), and India (IDD).