Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces MOCDA, a meta-learning framework for open compound domain adaptation in semantic segmentation, which clusters target domains, learns independent batch normalization, fuses predictions, and updates online for better generalization.

Contribution

The paper proposes a novel meta-learning based approach for OCDA that models target domains continuously and improves segmentation performance across unseen domains.

Findings

Achieves state-of-the-art results on synthetic-to-real benchmarks.

Effectively clusters target domains by style for better adaptation.

Online model updating enhances generalization to new domains.

Abstract

Open compound domain adaptation (OCDA) is a domain adaptation setting, where target domain is modeled as a compound of multiple unknown homogeneous domains, which brings the advantage of improved generalization to unseen domains. In this work, we propose a principled meta-learning based approach to OCDA for semantic segmentation, MOCDA, by modeling the unlabeled target domain continuously. Our approach consists of four key steps. First, we cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. Then, different sub-target domains are split into independent branches, for which batch normalization parameters are learnt to treat them independently. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. Meanwhile, we learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization. We validate the benefits of our approach by extensive experiments on synthetic-to-real knowledge transfer benchmark datasets, where we achieve the state-of-the-art performance in both compound and open domains.