Online Convex Optimization with Heavy Tails: Old Algorithms, New Regrets, and Applications

TL;DR

This paper investigates classical online convex optimization algorithms in heavy-tailed noise settings, establishing optimal regret bounds without modifications and applying these results to nonconvex optimization and broader scenarios.

Contribution

It provides the first optimal regret bounds for old algorithms in heavy-tailed stochastic gradients without needing gradient clipping.

Findings

Classical algorithms achieve optimal regret bounds in heavy-tailed settings.

First provable convergence results for nonconvex optimization under heavy-tailed noise.

Extensions to smooth OCO and optimistic algorithms for broader cases.

Abstract

In Online Convex Optimization (OCO), when the stochastic gradient has a finite variance, many algorithms provably work and guarantee a sublinear regret. However, limited results are known if the gradient estimate has a heavy tail, i.e., the stochastic gradient only admits a finite $\mathsf{p}$-th central moment for some $\mathsf{p}\in\left(1,2\right]$. Motivated by it, this work examines different old algorithms for OCO (e.g., Online Gradient Descent) in the more challenging heavy-tailed setting. Under the standard bounded domain assumption, we establish new regrets for these classical methods without any algorithmic modification. Remarkably, these regret bounds are fully optimal in all parameters (can be achieved even without knowing $\mathsf{p}$), suggesting that OCO with heavy tails can be solved effectively without any extra operation (e.g., gradient clipping). Our new results have several applications. A particularly interesting one is the first provable and optimal convergence result for nonsmooth nonconvex optimization under heavy-tailed noise without gradient clipping. Furthermore, we explore broader settings (e.g., smooth OCO) and extend our ideas to optimistic algorithms to handle different cases simultaneously.