On the Uniform Convergence of Subdifferentials in Stochastic Optimization and Learning

TL;DR

This paper establishes a reduction principle linking the uniform convergence of subdifferentials in stochastic optimization to simpler subgradient convergence, providing sharp rates and insights into nonsmooth empirical objectives.

Contribution

It introduces a general reduction principle for subdifferential convergence in stochastic optimization, enabling sharper analysis without differentiability assumptions.

Findings

Derived sharp uniform convergence rates for subdifferential mappings.

Reduced set-valued subdifferential convergence to vector-valued subgradient convergence.

Provided new insights into the geometry of nonsmooth empirical optimization problems.

Abstract

We investigate the uniform convergence of subdifferential mappings from empirical risk to population risk in nonsmooth, nonconvex stochastic optimization. This question is key to understanding how empirical stationary points approximate population ones, yet characterizing this convergence remains a fundamental challenge due to the set-valued and nonsmooth nature of subdifferentials. This work establishes a general reduction principle: for weakly convex stochastic objectives, over any open subset of the domain, we show that a uniform bound on the convergence of selected subgradients-chosen arbitrarily from subdifferential sets-yields a corresponding uniform bound on the Hausdorff distance between the subdifferentials. This deterministic result reduces the study of set-valued subdifferential convergence to simpler vector-valued subgradient convergence. We apply this reduction to derive sharp uniform convergence rates for subdifferential mappings in stochastic convex-composite optimization, without relying on differentiability assumptions on the population risk. These guarantees clarify the landscape of nonsmooth empirical objectives and offer new insight into the geometry of optimization problems arising in robust statistics and related applications.