Few-shot Domain Adaptation by Causal Mechanism Transfer

TL;DR

This paper introduces a novel few-shot domain adaptation method based on transfer of causal mechanisms, allowing adaptation across significantly different distributions by leveraging invariance in data generating processes.

Contribution

It proposes mechanism transfer as a new assumption for domain adaptation, utilizing causal structural equations to handle nonparametric distribution shifts.

Findings

The method is theoretically sound and effective in experiments.

It outperforms existing DA methods under complex distribution shifts.

First to fully leverage structural causal models for DA.

Abstract

We study few-shot supervised domain adaptation (DA) for regression problems, where only a few labeled target domain data and many labeled source domain data are available. Many of the current DA methods base their transfer assumptions on either parametrized distribution shift or apparent distribution similarities, e.g., identical conditionals or small distributional discrepancies. However, these assumptions may preclude the possibility of adaptation from intricately shifted and apparently very different distributions. To overcome this problem, we propose mechanism transfer, a meta-distributional scenario in which a data generating mechanism is invariant among domains. This transfer assumption can accommodate nonparametric shifts resulting in apparently different distributions while providing a solid statistical basis for DA. We take the structural equations in causal modeling as an example and propose a novel DA method, which is shown to be useful both theoretically and experimentally. Our method can be seen as the first attempt to fully leverage the structural causal models for DA.