Analysis of Kernel Mean Matching under Covariate Shift

TL;DR

This paper investigates the theoretical properties of Kernel Mean Matching (KMM) under covariate shift, providing confidence bounds and demonstrating its superiority over plug-in estimators in correcting sampling bias.

Contribution

The paper derives high probability bounds for KMM under covariate shift and compares it with plug-in estimators, establishing its effectiveness and theoretical advantages.

Findings

KMM has favorable convergence properties under covariate shift.

KMM outperforms plug-in estimators in bias correction.

The convergence rate depends on regularity and capacity measures.

Abstract

In real supervised learning scenarios, it is not uncommon that the training and test sample follow different probability distributions, thus rendering the necessity to correct the sampling bias. Focusing on a particular covariate shift problem, we derive high probability confidence bounds for the kernel mean matching (KMM) estimator, whose convergence rate turns out to depend on some regularity measure of the regression function and also on some capacity measure of the kernel. By comparing KMM with the natural plug-in estimator, we establish the superiority of the former hence provide concrete evidence/understanding to the effectiveness of KMM under covariate shift.