Nonparametric Regression for Random Unbiased Perturbations

TL;DR

This paper investigates nonparametric regression under random unbiased perturbations of the conditional distribution, revealing how such perturbations inflate variance, alter bandwidth selection, and impact statistical efficiency and limits.

Contribution

It introduces the concept of RUPs, derives an extended bias-variance decomposition including distributional variance, and establishes minimax bounds showing the fundamental impact of dataset-level perturbations.

Findings

Distributional uncertainty reduces effective sample size to n/(1 + nτ)

Optimal bandwidth scales as τ^{1/(2β+1)} under dominant distributional uncertainty

Minimax bounds demonstrate the fundamental limits imposed by RUPs

Abstract

We study nonparametric regression with covariates $X$ and outcome $Y$ under random unbiased perturbations (RUPs) of the conditional distribution $Y|X$, where the marginal distribution of covariates, $P^X$, remains fixed but the conditional law, $P^{Y|X}$, varies randomly across datasets. Unlike adversarial distribution shift frameworks that yield conservative worst-case guarantees, RUPs induce dataset-level variance inflation rather than systematic bias. We provide examples of RUPs and show that this distributional uncertainty reduces the effective sample size to $n_{\mathrm{eff}} = n/(1 + n \tau)$, where $\tau\in [0,1]$ quantifies the perturbation strength. For local polynomial estimators, we derive an extended bias-variance decomposition that includes a distributional variance term with the same bandwidth scaling as classical sampling variance. This leads to a modified bandwidth selection principle: when distributional uncertainty dominates sampling uncertainty ($\tau \gg 1/n$), optimal bandwidths scale as $\tau^{1/(2\beta+1)}$ rather than the usual $n^{-1/(2\beta+1)}$, where $\beta$ indicates the smoothness of the function class considered. We also establish matching minimax lower bounds showing that there exists an RUP for which this effective sample size $n_{\mathrm{eff}}$ is fundamental. Our results demonstrate that random dataset-level perturbations create a distinct mode of uncertainty that affects both practical tuning and fundamental statistical limits.