Duality-free Methods for Stochastic Composition Optimization

TL;DR

This paper introduces duality-free stochastic composition methods combining variance reduction techniques to efficiently optimize complex composition problems common in machine learning, achieving linear convergence.

Contribution

It develops a novel duality-free approach with variance reduction for stochastic composition optimization, applicable to convex and non-convex outer functions.

Findings

Achieves linear convergence rate for convex composition problems.

Effective in non-convex outer function scenarios with strong convexity.

Experimental results demonstrate the methods' efficiency.

Abstract

We consider the composition optimization with two expected-value functions in the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement learning and nonlinear embedding}. Full Gradient or classical stochastic gradient descent based optimization algorithms are unsuitable or computationally expensive to solve this problem due to the inner expectation $\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x)$. We propose a duality-free based stochastic composition method that combines variance reduction methods to address the stochastic composition problem. We apply SVRG and SAGA based methods to estimate the inner function, and duality-free method to estimate the outer function. We prove the linear convergence rate not only for the convex composition problem, but also for the case that the individual outer functions are non-convex while the objective function is strongly-convex. We also provide the results of experiments that show the effectiveness of our proposed methods.