COD: Learning Conditional Invariant Representation for Domain Adaptation Regression

TL;DR

This paper introduces a novel theoretical framework and a new discrepancy measure, COD, for domain adaptation regression, enabling better generalization across domains with continuous outputs.

Contribution

It establishes the sufficiency theory for regression models in domain adaptation and proposes the Conditional Operator Discrepancy (COD) for continuous conditional distribution alignment.

Findings

Theoretical validation of the sufficiency of cross-domain conditional discrepancy.

Introduction of COD as a metric for continuous conditional distributions.

Experimental results show superior performance over state-of-the-art methods.

Abstract

Aiming to generalize the label knowledge from a source domain with continuous outputs to an unlabeled target domain, Domain Adaptation Regression (DAR) is developed for complex practical learning problems. However, due to the continuity problem in regression, existing conditional distribution alignment theory and methods with discrete prior, which are proven to be effective in classification settings, are no longer applicable. In this work, focusing on the feasibility problems in DAR, we establish the sufficiency theory for the regression model, which shows the generalization error can be sufficiently dominated by the cross-domain conditional discrepancy. Further, to characterize conditional discrepancy with continuous conditioning variable, a novel Conditional Operator Discrepancy (COD) is proposed, which admits the metric property on conditional distributions via the kernel embedding theory. Finally, to minimize the discrepancy, a COD-based conditional invariant representation learning model is proposed, and the reformulation is derived to show that reasonable modifications on moment statistics can further improve the discriminability of the adaptation model. Extensive experiments on standard DAR datasets verify the validity of theoretical results and the superiority over SOTA DAR methods.