Confidence intervals for nonparametric regression

TL;DR

This paper derives nonasymptotic probabilistic bounds for nonparametric regression costs using Rademacher and VC theories, addressing both independent and dependent data scenarios.

Contribution

It introduces new nonasymptotic bounds for regression error and discusses their optimality, extending analysis to dependent data cases.

Findings

Nonasymptotic bounds established for regression schemes.

Analysis includes independent and dependent training samples.

Results demonstrate bounds' optimality in L^2-distance.

Abstract

We demonstrate and discuss nonasymptotic bounds in probability for the cost of a regression scheme with a general loss function from the perspective of the Rademacher theory, and for the optimality with respect to the average $L^{2}$-distance to the underlying conditional expectations of least squares regression outcomes from the perspective of the Vapnik-Chervonenkis theory. The results follow from an analysis involving independent but possibly nonstationary training samples and can be extended, in a manner that we explain and illustrate, to relevant cases in which the training sample exhibits dependence.