Beyond Sharing Weights for Deep Domain Adaptation

TL;DR

This paper proposes a two-stream deep learning architecture for domain adaptation that explicitly models domain shifts, outperforming shared-weight methods in object recognition and detection tasks.

Contribution

It introduces a novel two-stream architecture with related but not shared weights, improving domain adaptation performance over existing shared-weight approaches.

Findings

Higher accuracy than state-of-the-art methods

Outperforms shared-weight networks in supervised settings

Consistently better in unsupervised domain adaptation

Abstract

The performance of a classifier trained on data coming from a specific domain typically degrades when applied to a related but different one. While annotating many samples from the new domain would address this issue, it is often too expensive or impractical. Domain Adaptation has therefore emerged as a solution to this problem; It leverages annotated data from a source domain, in which it is abundant, to train a classifier to operate in a target domain, in which it is either sparse or even lacking altogether. In this context, the recent trend consists of learning deep architectures whose weights are shared for both domains, which essentially amounts to learning domain invariant features. Here, we show that it is more effective to explicitly model the shift from one domain to the other. To this end, we introduce a two-stream architecture, where one operates in the source domain and the other in the target domain. In contrast to other approaches, the weights in corresponding layers are related but not shared. We demonstrate that this both yields higher accuracy than state-of-the-art methods on several object recognition and detection tasks and consistently outperforms networks with shared weights in both supervised and unsupervised settings.