Accelerating Stochastic Composition Optimization

TL;DR

This paper introduces ASC-PG, a novel accelerated stochastic proximal gradient method for stochastic composition optimization, capable of handling nonsmooth penalties and demonstrating faster convergence and optimal sample complexity.

Contribution

The paper presents the first proximal gradient method for stochastic composition problems that handles nonsmooth regularization and achieves faster convergence.

Findings

ASC-PG outperforms existing algorithms in convergence speed.

ASC-PG achieves optimal sample-error complexity in key cases.

Numerical experiments validate the effectiveness of ASC-PG.

Abstract

Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using two different timescales. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We further demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.