Nonparametric Regression, Confidence Regions and Regularization

TL;DR

This paper introduces a unified, nonparametric regression framework on the unit interval using a universal confidence region defined by linear inequalities, enabling honest, non-asymptotic inference based on shape and smoothness constraints.

Contribution

It proposes a novel approach to nonparametric regression that combines confidence regions with regularization based on shape and smoothness, providing honest, non-asymptotic confidence bounds.

Findings

Defines a universal confidence region using linear inequalities.

Allows regularization based on shape and smoothness.

Provides honest, non-asymptotic confidence bounds.

Abstract

In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and non-asymptotic confidence region which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically centre on certain simplest functions in that region where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the confidence region can be used to provide honest non-asymptotic confidence bounds which are less informative but conceptually much simpler.