A Minimax Theory of Nonparametric Regression Under Covariate Shift

TL;DR

This paper develops a minimax theoretical framework for nonparametric regression under covariate shift, introducing the transfer function and analyzing various regimes of convergence rates, including unbounded covariate support.

Contribution

It introduces the transfer function concept and characterizes minimax rates under covariate shift, covering cases with unbounded covariate support and multiple regimes.

Findings

Rates include classical, better source/target, and multiplicative regimes.

Achieves rates up to logarithmic factors with a design-adaptive estimator.

Extends theory to unbounded covariate support.

Abstract

We consider nonparametric regression under covariate shift, where we observe samples from both the target distribution and a related but distinct source distribution. We introduce a novel object, the transfer function, and show that properties of its domain determine our minimax rates. Those exhibit a variety of regimes, including classical rates, governed by the better of source-only and target-only rates, as well as regimes in which the convergence rates exhibit multiplicative interactions between the sample sizes and are faster than the best-of-two benchmark. The rates are shown to be achieved up to logarithmic factors by a design-adaptive estimator. Compared with existing theory, our results cover the case in which covariates have unbounded support.