Gradient-Variation Online Learning under Generalized Smoothness

TL;DR

This paper advances gradient-variation online learning by extending algorithms to generalized smoothness conditions, enabling adaptive, fast convergence in diverse optimization scenarios without prior curvature knowledge.

Contribution

It introduces a generalized smoothness framework, extends optimistic mirror descent, and designs a universal, adaptive algorithm with optimal regret bounds for convex and strongly convex functions.

Findings

Achieves gradient-variation regret bounds under generalized smoothness.

Develops a Lipschitz-adaptive meta-algorithm for unbounded gradients.

Provides applications for fast convergence in games and stochastic optimization.

Abstract

Gradient-variation online learning aims to achieve regret guarantees that scale with variations in the gradients of online functions, which has been shown to be crucial for attaining fast convergence in games and robustness in stochastic optimization, hence receiving increased attention. Existing results often require the smoothness condition by imposing a fixed bound on gradient Lipschitzness, which may be unrealistic in practice. Recent efforts in neural network optimization suggest a generalized smoothness condition, allowing smoothness to correlate with gradient norms. In this paper, we systematically study gradient-variation online learning under generalized smoothness. We extend the classic optimistic mirror descent algorithm to derive gradient-variation regret by analyzing stability over the optimization trajectory and exploiting smoothness locally. Then, we explore universal online learning, designing a single algorithm with the optimal gradient-variation regrets for convex and strongly convex functions simultaneously, without requiring prior knowledge of curvature. This algorithm adopts a two-layer structure with a meta-algorithm running over a group of base-learners. To ensure favorable guarantees, we design a new Lipschitz-adaptive meta-algorithm, capable of handling potentially unbounded gradients while ensuring a second-order bound to effectively ensemble the base-learners. Finally, we provide the applications for fast-rate convergence in games and stochastic extended adversarial optimization.