A Short Note of PAGE: Optimal Convergence Rates for Nonconvex Optimization

TL;DR

This note revisits the PAGE algorithm for nonconvex optimization, providing a clear convergence analysis that establishes optimal rates and facilitates adaptation to other algorithms.

Contribution

It offers a simplified, comprehensive convergence analysis of PAGE, demonstrating its optimal rates and adaptability for future research.

Findings

PAGE achieves optimal convergence rates for nonconvex problems

Analysis is simplified and easily adaptable to other algorithms

Provides insights for future optimization research

Abstract

In this note, we first recall the nonconvex problem setting and introduce the optimal PAGE algorithm (Li et al., ICML'21). Then we provide a simple and clean convergence analysis of PAGE for achieving optimal convergence rates. Moreover, PAGE and its analysis can be easily adopted and generalized to other works. We hope that this note provides the insights and is helpful for future works.