Unsupervised Domain Adaptation with Residual Transfer Networks

TL;DR

This paper introduces a novel deep learning approach for unsupervised domain adaptation that learns residual functions to adapt classifiers and features, outperforming existing methods on benchmark datasets.

Contribution

It proposes a residual transfer network that relaxes shared-classifier assumptions and explicitly models classifier differences for improved domain adaptation.

Findings

Outperforms state-of-the-art methods on standard benchmarks.

Effectively learns adaptive classifiers and transferable features.

Can be integrated into most feed-forward deep networks.

Abstract

The recent success of deep neural networks relies on massive amounts of labeled data. For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. In this paper, we propose a new approach to domain adaptation in deep networks that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. We relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function. We enable classifier adaptation by plugging several layers into deep network to explicitly learn the residual function with reference to the target classifier. We fuse features of multiple layers with tensor product and embed them into reproducing kernel Hilbert spaces to match distributions for feature adaptation. The adaptation can be achieved in most feed-forward models by extending them with new residual layers and loss functions, which can be trained efficiently via back-propagation. Empirical evidence shows that the new approach outperforms state of the art methods on standard domain adaptation benchmarks.