mDALU: Multi-Source Domain Adaptation and Label Unification with Partial Datasets

TL;DR

This paper introduces mDALU, a novel approach for multi-source domain adaptation and label unification that effectively handles partial annotations and different data modalities, improving performance across various vision tasks.

Contribution

It proposes a two-stage adaptation method with modules to mitigate negative transfer and unify labels, advancing multi-source domain adaptation with partial datasets.

Findings

Outperforms existing methods significantly on image classification.

Achieves superior results in 2D semantic segmentation.

Effective in joint 2D-3D semantic segmentation tasks.

Abstract

One challenge of object recognition is to generalize to new domains, to more classes and/or to new modalities. This necessitates methods to combine and reuse existing datasets that may belong to different domains, have partial annotations, and/or have different data modalities. This paper formulates this as a multi-source domain adaptation and label unification problem, and proposes a novel method for it. Our method consists of a partially-supervised adaptation stage and a fully-supervised adaptation stage. In the former, partial knowledge is transferred from multiple source domains to the target domain and fused therein. Negative transfer between unmatching label spaces is mitigated via three new modules: domain attention, uncertainty maximization and attention-guided adversarial alignment. In the latter, knowledge is transferred in the unified label space after a label completion process with pseudo-labels. Extensive experiments on three different tasks - image classification, 2D semantic image segmentation, and joint 2D-3D semantic segmentation - show that our method outperforms all competing methods significantly.