Improved Multi-Source Domain Adaptation by Preservation of Factors

TL;DR

This paper introduces Factor-Preserving Domain Adaptation (FP-DA), a novel method that maintains task-relevant factors during multi-source domain adaptation to improve image classification performance across diverse domains.

Contribution

The paper proposes FP-DA, a new adversarial training approach that preserves important domain factors, addressing negative transfer caused by removing all domain-specific features.

Findings

FP-DA achieves higher average performance across multiple domains.

Preserving task-relevant factors prevents negative transfer.

Standard transfer experiments help identify domain factors.

Abstract

Domain Adaptation (DA) is a highly relevant research topic when it comes to image classification with deep neural networks. Combining multiple source domains in a sophisticated way to optimize a classification model can improve the generalization to a target domain. Here, the difference in data distributions of source and target image datasets plays a major role. In this paper, we describe based on a theory of visual factors how real-world scenes appear in images in general and how recent DA datasets are composed of such. We show that different domains can be described by a set of so called domain factors, whose values are consistent within a domain, but can change across domains. Many DA approaches try to remove all domain factors from the feature representation to be domain invariant. In this paper we show that this can lead to negative transfer since task-informative factors can get lost as well. To address this, we propose Factor-Preserving DA (FP-DA), a method to train a deep adversarial unsupervised DA model, which is able to preserve specific task relevant factors in a multi-domain scenario. We demonstrate on CORe50, a dataset with many domains, how such factors can be identified by standard one-to-one transfer experiments between single domains combined with PCA. By applying FP-DA, we show that the highest average and minimum performance can be achieved.