Boosting Accelerated Proximal Gradient Method with Adaptive Sampling for Stochastic Composite Optimization

TL;DR

This paper introduces adaNAPG, an adaptive accelerated proximal gradient method that improves stochastic composite optimization by combining Nesterov acceleration with adaptive sampling, achieving optimal complexity and convergence properties.

Contribution

The paper proposes adaNAPG, a novel adaptive sampling variant of Nesterov accelerated proximal gradient, with proven optimal complexity and a new central limit theorem for convergence analysis.

Findings

Achieves optimal iteration and sample complexity.

Establishes a central limit theorem for adaNAPG.

Demonstrates improved convergence rate and efficiency.

Abstract

We develop an adaptive Nesterov accelerated proximal gradient (adaNAPG) algorithm for stochastic composite optimization problems, boosting the Nesterov accelerated proximal gradient (NAPG) algorithm through the integration of an adaptive sampling strategy for gradient estimation. We provide a complexity analysis demonstrating that the new algorithm, adaNAPG, achieves both the optimal iteration complexity and the optimal sample complexity as outlined in the existing literature. Additionally, we establish a central limit theorem for the iteration sequence of the new algorithm adaNAPG, elucidating its convergence rate and efficiency.