Theoretic Analysis and Extremely Easy Algorithms for Domain Adaptive Feature Learning

TL;DR

This paper provides a theoretical analysis of feature learning in domain adaptation, highlighting the importance of matching second moments of source and target distributions, and introduces a simple, effective algorithm extended to deep linear models.

Contribution

It offers a new theoretical understanding of domain adaptation with linear classifiers and proposes an extremely simple, yet effective, feature learning algorithm for this purpose.

Findings

Matching second moments improves adaptation performance

The proposed algorithm is simple and effective

Deep linear models enhance domain adaptation

Abstract

Domain adaptation problems arise in a variety of applications, where a training dataset from the \textit{source} domain and a test dataset from the \textit{target} domain typically follow different distributions. The primary difficulty in designing effective learning models to solve such problems lies in how to bridge the gap between the source and target distributions. In this paper, we provide comprehensive analysis of feature learning algorithms used in conjunction with linear classifiers for domain adaptation. Our analysis shows that in order to achieve good adaptation performance, the second moments of the source domain distribution and target domain distribution should be similar. Based on our new analysis, a novel extremely easy feature learning algorithm for domain adaptation is proposed. Furthermore, our algorithm is extended by leveraging multiple layers, leading to a deep linear model. We evaluate the effectiveness of the proposed algorithms in terms of domain adaptation tasks on the Amazon review dataset and the spam dataset from the ECML/PKDD 2006 discovery challenge.