Graphical Convergence of Subgradients in Nonconvex Optimization and Learning

TL;DR

This paper studies the convergence properties of subgradients in nonconvex, weakly convex optimization problems, providing new theoretical rates for subgradient estimation and analyzing the landscape of robust nonlinear regression.

Contribution

It introduces dimension-dependent and independent convergence rates for subgradient estimation in weakly convex problems, advancing understanding of nonsmooth nonconvex optimization landscapes.

Findings

Dimension-dependent subgradient estimation rates established

Dimension-independent rates for generalized linear models

Analysis of nonsmooth landscape in robust nonlinear regression

Abstract

We investigate the stochastic optimization problem of minimizing population risk, where the loss defining the risk is assumed to be weakly convex. Compositions of Lipschitz convex functions with smooth maps are the primary examples of such losses. We analyze the estimation quality of such nonsmooth and nonconvex problems by their sample average approximations. Our main results establish dimension-dependent rates on subgradient estimation in full generality and dimension-independent rates when the loss is a generalized linear model. As an application of the developed techniques, we analyze the nonsmooth landscape of a robust nonlinear regression problem.