Minimax Optimal Estimation of Convex Functions in the Supreme Norm

TL;DR

This paper establishes the first rigorous minimax optimal rate of convergence for estimating convex functions in the supremum norm, using a novel combination of information theory and penalized spline methods.

Contribution

It introduces a new approach to derive the minimax rate for convex function estimation under the supremum norm, combining lower bounds with a novel penalized spline estimator.

Findings

Established the minimax lower bound using piecewise quadratic functions.

Developed a penalized convex spline estimator achieving the minimax upper bound.

Proved the optimality of the estimator in the supremum norm for convex functions.

Abstract

Estimation of convex functions finds broad applications in engineering and science, while convex shape constraint gives rise to numerous challenges in asymptotic performance analysis. This paper is devoted to minimax optimal estimation of univariate convex functions from the H\"older class in the framework of shape constrained nonparametric estimation. Particularly, the paper establishes the optimal rate of convergence in two steps for the minimax sup-norm risk of convex functions with the H\"older order between one and two. In the first step, by applying information theoretical results on probability measure distance, we establish the minimax lower bound under the supreme norm by constructing a novel family of piecewise quadratic convex functions in the H\"older class. In the second step, we develop a penalized convex spline estimator and establish the minimax upper bound under the supreme norm. Due to the convex shape constraint, the optimality conditions of penalized convex splines are characterized by nonsmooth complementarity conditions. By exploiting complementarity methods, a critical uniform Lipschitz property of optimal spline coefficients in the infinity norm is established. This property, along with asymptotic estimation techniques, leads to uniform bounds for bias and stochastic errors on the entire interval of interest. This further yields the optimal rate of convergence by choosing the suitable number of knots and penalty value. The present paper provides the first rigorous justification of the optimal minimax risk for convex estimation under the supreme norm.