On rates of convergence for sample average approximations without smoothness

TL;DR

This paper develops a convergence rate theory for sample average approximation in stochastic optimization without requiring smoothness or continuity assumptions, applicable to various complex models.

Contribution

It extends SAA convergence analysis to non-smooth, discontinuous, and non-Lipschitz settings using a tame-topological framework, broadening applicability.

Findings

Uniform control of empirical processes yields convergence rates for optimal values.

Weak limits for empirical optimal values are established at the n^{-1/2} scale.

Framework applies to classification, neural networks, and non-Lipschitz objectives.

Abstract

Sample average approximation (SAA) replaces an intractable expected objective by an empirical average and is a basic device of modern stochastic optimization. We develop a rate theory for optimal values and empirical $\varepsilon$-minimizers that does not assume continuity, lower semicontinuity, or smooth perturbation structure of the sample objectives. Working on $\ell^{\infty}(X)$ with the Hoffmann--J{\o}rgensen outer-probability formalism, we show that uniform control of the empirical objective process transfers deterministically to convergence rates for optimal values, excess risks of empirical $\varepsilon$-minimizers, and, under a sharp-growth condition, distances to the expected objective solution set. Combined with the directional differentiability of the infimum functional, this yields weak limits for empirical optimal values at the $n^{-1/2}$ scale. Combined with LILs and maximal inequalities, it yields outer almost-sure and outer-mean rates. The definability, envelope, and VC-subgraph hypotheses are verified for definable discontinuous or non-Lipschitz classes arising in direct $0$--$1$ classification, fixed-architecture neural networks, threshold regression, and non-Lipschitz $\ell_{p}$-type objectives with rational $0<p<1$. Practical sufficient conditions for measurability hypotheses are discussed. Together, the framework extends continuity-based SAA theory to a tame-topological setting.