# The importance of better models in stochastic optimization

## TL;DR

This paper introduces the aProx family of models for stochastic optimization, enhancing stability, convergence, and robustness across diverse problem classes, including non-convex objectives, with theoretical guarantees and empirical validation.

## Contribution

It proposes the aProx modeling framework that improves robustness and convergence in stochastic optimization, extending to weakly convex and non-convex problems with theoretical and experimental support.

## Key findings

- aProx models improve stability and convergence
- Robustness extends to non-convex and weakly convex objectives
- Experimental results show reduced sensitivity and faster convergence

## Abstract

Standard stochastic optimization methods are brittle, sensitive to stepsize choices and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic minimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models---which we call the aProx family---stochastic methods can be made stable, provably convergent and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning applications. We highlight the importance of robustness and accurate modeling with a careful experimental evaluation of convergence time and algorithm sensitivity.