Convergence Rates for Kernel Regression in Infinite Dimensional Spaces

TL;DR

This paper establishes the optimal convergence rates for kernel regression when the covariate space is infinite dimensional, highlighting unique challenges and solutions in such settings.

Contribution

It derives the optimal convergence rates for kernel estimates in infinite dimensional spaces and introduces an adaptive bandwidth selection method.

Findings

Optimal convergence rate derived for infinite dimensional covariates.

Small ball probability influences asymptotic variance.

Adaptive bandwidth selection improves estimation accuracy.

Abstract

We consider a nonparametric regression setup, where the covariate is a random element in a complete separable metric space, and the parameter of interest associated with the conditional distribution of the response lies in a separable Banach space. We derive the optimum convergence rate for the kernel estimate of the parameter in this setup. The small ball probability in the covariate space plays a critical role in determining the asymptotic variance of kernel estimates. Unlike the case of finite dimensional covariates, we show that the asymptotic orders of the bias and the variance of the estimate achieving the optimum convergence rate may be different for infinite dimensional covariates. Also, the bandwidth, which balances the bias and the variance, may lead to an estimate with suboptimal mean square error for infinite dimensional covariates. We describe a data-driven adaptive choice of the bandwidth, and derive the asymptotic behavior of the adaptive estimate.