LRS-DAG: Low Resource Supervised Domain Adaptation with Generalization Across Domains

TL;DR

LRS-DAG is a novel low-resource supervised domain adaptation method that maintains source domain performance while effectively adapting to target domains, especially in low-resource settings, by learning domain mappings with encoder layers.

Contribution

The paper introduces LRS-DAG, a new algorithm that preserves source domain performance during adaptation and proposes a novel metric for domain mapping in low-resource scenarios.

Findings

LRS-DAG performs comparably to fine-tuning on MNIST to SVHN transfer.

LRS-DAG outperforms fine-tuning on synthetic datasets similar to MNIST.

The method maintains source domain performance while adapting to target domains.

Abstract

Current state of the art methods in Domain Adaptation follow adversarial approaches, making training a challenge. Existing non-adversarial methods learn mappings between the source and target domains, to achieve reasonable performance. However, even these methods do not focus on a key aspect: maintaining performance on the source domain, even after optimizing over the target domain. Additionally, there exist very few methods in low resource supervised domain adaptation. This work proposes a method, LRS-DAG, that aims to solve these current issues in the field. By adding a set of "encoder layers" which map the target domain to the source, and can be removed when dealing directly with the source data, the model learns to perform optimally on both domains. LRS-DAG showcases its uniqueness by being a new algorithm for low resource domain adaptation which maintains performance over the source domain, with a new metric for learning mappings between domains being introduced. We show that, in the case of FCNs, when transferring from MNIST to SVHN, LRS-DAG performs comparably to fine tuning, with the advantage of maintaining performance over the source domain. LRS-DAG outperforms fine tuning when transferring to a synthetic dataset similar to MNIST, which is a setting more representative of low resource supervised domain adaptation.