Conceptual Domain Adaptation Using Deep Learning

TL;DR

This paper introduces a new framework for conceptual domain adaptation in deep learning, focusing on aligning high-level representations across domains to improve transfer learning when low-level features differ.

Contribution

It proposes a search framework for aligning high-level concepts in deep networks, addressing limitations of traditional methods that assume shared hidden units.

Findings

Framework improves alignment of high-level representations

Addresses limitations of traditional domain adaptation methods

Enhances transfer learning across diverse domains

Abstract

Deep learning has recently been shown to be instrumental in the problem of domain adaptation, where the goal is to learn a model on a target domain using a similar --but not identical-- source domain. The rationale for coupling both techniques is the possibility of extracting common concepts across domains. Considering (strictly) local representations, traditional deep learning assumes common concepts must be captured in the same hidden units. We contend that jointly training a model with source and target data using a single deep network is prone to failure when there is inherently lower-level representational discrepancy between the two domains; such discrepancy leads to a misalignment of corresponding concepts in separate hidden units. We introduce a search framework to correctly align high-level representations when training deep networks; such framework leads to the notion of conceptual --as opposed to representational-- domain adaptation.