On Deep Domain Adaptation: Some Theoretical Understandings

TL;DR

This paper provides a theoretical framework explaining how deep domain adaptation can effectively bridge source and target domains in a joint space, supporting its empirical success with formal bounds on transfer learning loss.

Contribution

It introduces the first theoretical analysis that characterizes the bounds on transfer loss in deep domain adaptation, explaining its effectiveness.

Findings

Provides a rigorous loss bound for transfer learning in deep domain adaptation

Explains why closing the domain gap in joint space reduces transfer loss

Generalizes existing empirical results with theoretical support

Abstract

Compared with shallow domain adaptation, recent progress in deep domain adaptation has shown that it can achieve higher predictive performance and stronger capacity to tackle structural data (e.g., image and sequential data). The underlying idea of deep domain adaptation is to bridge the gap between source and target domains in a joint space so that a supervised classifier trained on labeled source data can be nicely transferred to the target domain. This idea is certainly intuitive and powerful, however, limited theoretical understandings have been developed to support its underpinning principle. In this paper, we have provided a rigorous framework to explain why it is possible to close the gap of the target and source domains in the joint space. More specifically, we first study the loss incurred when performing transfer learning from the source to the target domain. This provides a theory that explains and generalizes existing work in deep domain adaptation which was mainly empirical. This enables us to further explain why closing the gap in the joint space can directly minimize the loss incurred for transfer learning between the two domains. To our knowledge, this offers the first theoretical result that characterizes a direct bound on the joint space and the gain of transfer learning via deep domain adaptation