General regularization in covariate shift adaptation

TL;DR

This paper reviews and extends error bounds for sample reweighting in kernel regression under covariate shift, demonstrating that fewer samples are needed for accurate learning when data distributions differ.

Contribution

It combines known bounds to derive novel results showing reduced sample complexity under weak smoothness conditions in covariate shift scenarios.

Findings

Fewer samples are required for accurate kernel regression under covariate shift.

New error bounds improve upon previous analyses.

Results hold under weak smoothness assumptions.

Abstract

Sample reweighting is one of the most widely used methods for correcting the error of least squares learning algorithms in reproducing kernel Hilbert spaces (RKHS), that is caused by future data distributions that are different from the training data distribution. In practical situations, the sample weights are determined by values of the estimated Radon-Nikod\'ym derivative, of the future data distribution w.r.t.~the training data distribution. In this work, we review known error bounds for reweighted kernel regression in RKHS and obtain, by combination, novel results. We show under weak smoothness conditions, that the amount of samples, needed to achieve the same order of accuracy as in the standard supervised learning without differences in data distributions, is smaller than proven by state-of-the-art analyses.