Generalization Bounds for Domain Adaptation

TL;DR

This paper introduces a new theoretical framework for deriving generalization bounds in domain adaptation, analyzing convergence, and factors influencing learning performance, supported by numerical experiments.

Contribution

It develops novel bounds using integral probability metrics and extends classical inequalities to analyze asymptotic convergence in domain adaptation.

Findings

Derived new generalization bounds for domain adaptation.

Analyzed asymptotic convergence and convergence rates.

Numerical experiments support theoretical results.

Abstract

In this paper, we provide a new framework to obtain the generalization bounds of the learning process for domain adaptation, and then apply the derived bounds to analyze the asymptotical convergence of the learning process. Without loss of generality, we consider two kinds of representative domain adaptation: one is with multiple sources and the other is combining source and target data. In particular, we use the integral probability metric to measure the difference between two domains. For either kind of domain adaptation, we develop a related Hoeffding-type deviation inequality and a symmetrization inequality to achieve the corresponding generalization bound based on the uniform entropy number. We also generalized the classical McDiarmid's inequality to a more general setting where independent random variables can take values from different domains. By using this inequality, we then obtain generalization bounds based on the Rademacher complexity. Afterwards, we analyze the asymptotic convergence and the rate of convergence of the learning process for such kind of domain adaptation. Meanwhile, we discuss the factors that affect the asymptotic behavior of the learning process and the numerical experiments support our theoretical findings as well.