Learning Smooth Representation for Unsupervised Domain Adaptation

TL;DR

This paper investigates how Lipschitz constraints can improve unsupervised domain adaptation by reducing error bounds, introducing a local smooth discrepancy measure, and optimizing factors like sample size and batch dimensions, leading to better performance on large datasets.

Contribution

It provides a mathematical analysis linking Lipschitz constraints to error bounds in domain adaptation and proposes an optimization strategy considering sample size, dimension, and batch size.

Findings

Lipschitzness reduces the error bound in domain adaptation.

The proposed model outperforms existing methods on standard benchmarks.

Sample size, dimension, and batch size significantly affect Lipschitz-based adaptation methods.

Abstract

Typical adversarial-training-based unsupervised domain adaptation methods are vulnerable when the source and target datasets are highly-complex or exhibit a large discrepancy between their data distributions. Recently, several Lipschitz-constraint-based methods have been explored. The satisfaction of Lipschitz continuity guarantees a remarkable performance on a target domain. However, they lack a mathematical analysis of why a Lipschitz constraint is beneficial to unsupervised domain adaptation and usually perform poorly on large-scale datasets. In this paper, we take the principle of utilizing a Lipschitz constraint further by discussing how it affects the error bound of unsupervised domain adaptation. A connection between them is built and an illustration of how Lipschitzness reduces the error bound is presented. A \textbf{local smooth discrepancy} is defined to measure Lipschitzness of a target distribution in a pointwise way. When constructing a deep end-to-end model, to ensure the effectiveness and stability of unsupervised domain adaptation, three critical factors are considered in our proposed optimization strategy, i.e., the sample amount of a target domain, dimension and batchsize of samples. Experimental results demonstrate that our model performs well on several standard benchmarks. Our ablation study shows that the sample amount of a target domain, the dimension and batchsize of samples indeed greatly impact Lipschitz-constraint-based methods' ability to handle large-scale datasets. Code is available at https://github.com/CuthbertCai/SRDA.