Approximations of Semicontinuous Functions with Applications to Stochastic Optimization and Statistical Estimation

TL;DR

This paper develops methods to approximate upper semicontinuous functions with piecewise affine functions, enabling better solutions for stochastic optimization and statistical estimation problems, with theoretical bounds and practical algorithms.

Contribution

It introduces a new approximation scheme for usc functions using hypo-converging piecewise affine functions and analyzes their complexity via covering numbers, with applications to stochastic optimization.

Findings

Approximation of usc functions by piecewise affine functions is feasible.

Upper and lower bounds on covering numbers for usc functions are established.

Confidence regions for solutions in stochastic optimization are derived with convergence rates.

Abstract

Upper semicontinuous (usc) functions arise in the analysis of maximization problems, distributionally robust optimization, and function identification, which includes many problems of nonparametric statistics. We establish that every usc function is the limit of a hypo-converging sequence of piecewise affine functions of the difference-of-max type and illustrate resulting algorithmic possibilities in the context of approximate solution of infinite-dimensional optimization problems. In an effort to quantify the ease with which classes of usc functions can be approximated by finite collections, we provide upper and lower bounds on covering numbers for bounded sets of usc functions under the Attouch-Wets distance. The result is applied in the context of stochastic optimization problems defined over spaces of usc functions. We establish confidence regions for optimal solutions based on sample average approximations and examine the accompanying rates of convergence. Examples from nonparametric statistics illustrate the results.