Sharp oracle inequalities for Least Squares estimators in shape restricted regression

TL;DR

This paper establishes sharp oracle inequalities for Least Squares estimators in shape-restricted regression, showing adaptive risk bounds and rates for isotonic, unimodal, and convex regression, including worst-case design scenarios.

Contribution

It provides new sharp oracle inequalities and adaptive risk bounds for LS estimators in shape-restricted regression, highlighting the impact of design points on convergence rates.

Findings

LS estimators achieve nonparametric rate $n^{-2/3}$ in isotonic and unimodal regression.

In convex regression, the rate can be as slow as $n^{-2/3}$ for worst-case designs.

Adaptive risk bounds depend on the number of pieces in the true function.

Abstract

The performance of Least Squares (LS) estimators is studied in isotonic, unimodal and convex regression. Our results have the form of sharp oracle inequalities that account for the model misspecification error. In isotonic and unimodal regression, the LS estimator achieves the nonparametric rate $n^{-2/3}$ as well as a parametric rate of order $k/n$ up to logarithmic factors, where $k$ is the number of constant pieces of the true parameter. In univariate convex regression, the LS estimator satisfies an adaptive risk bound of order $q/n$ up to logarithmic factors, where $q$ is the number of affine pieces of the true regression function. This adaptive risk bound holds for any design points. While Guntuboyina and Sen (2013) established that the nonparametric rate of convex regression is of order $n^{-4/5}$ for equispaced design points, we show that the nonparametric rate of convex regression can be as slow as $n^{-2/3}$ for some worst-case design points. This phenomenon can be explained as follows: Although convexity brings more structure than unimodality, for some worst-case design points this extra structure is uninformative and the nonparametric rates of unimodal regression and convex regression are both $n^{-2/3}$.