Projection-Free Algorithm for Stochastic Bi-level Optimization

TL;DR

This paper introduces the first projection-free stochastic bi-level optimization algorithm, SBFW, with improved sample complexity rates, applicable to streaming data and complex problems like matrix completion and reinforcement learning.

Contribution

It proposes the SBFW algorithm for stochastic bi-level optimization without projections and derives improved sample complexity rates, including for the special case of stochastic compositional problems.

Findings

SBFW achieves $ ilde{O}(rac{1}{ ext{epsilon}^3})$ complexity for convex objectives.

SCFW attains $ ilde{O}(rac{1}{ ext{epsilon}^2})$ complexity for convex problems.

Algorithms are effective for matrix completion and policy evaluation tasks.

Abstract

This work presents the first projection-free algorithm to solve stochastic bi-level optimization problems, where the objective function depends on the solution of another stochastic optimization problem. The proposed $\textbf{S}$tochastic $\textbf{Bi}$-level $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SBFW}$) algorithm can be applied to streaming settings and does not make use of large batches or checkpoints. The sample complexity of SBFW is shown to be $\mathcal{O}(\epsilon^{-3})$ for convex objectives and $\mathcal{O}(\epsilon^{-4})$ for non-convex objectives. Improved rates are derived for the stochastic compositional problem, which is a special case of the bi-level problem, and entails minimizing the composition of two expected-value functions. The proposed $\textbf{S}$tochastic $\textbf{C}$ompositional $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SCFW}$) is shown to achieve a sample complexity of $\mathcal{O}(\epsilon^{-2})$ for convex objectives and $\mathcal{O}(\epsilon^{-3})$ for non-convex objectives, at par with the state-of-the-art sample complexities for projection-free algorithms solving single-level problems. We demonstrate the advantage of the proposed methods by solving the problem of matrix completion with denoising and the problem of policy value evaluation in reinforcement learning.