A Simple Linear Convergence Analysis of the Point-SAGA Algorithm

TL;DR

This paper provides a straightforward proof of linear convergence for the Point-SAGA algorithm, a randomized method for convex optimization, extended to multiple prox calls per iteration, under smoothness and strong convexity assumptions.

Contribution

It generalizes Point-SAGA to multiple prox calls per iteration and offers a simple proof of its linear convergence under specific conditions.

Findings

Proves linear convergence of generalized Point-SAGA

Extends the algorithm to multiple prox calls per iteration

Simplifies the convergence proof

Abstract

Point-SAGA is a randomized algorithm for minimizing a sum of convex functions using their proximity operators (proxs), proposed by Defazio (2016). At every iteration, the prox of only one randomly chosen function is called. We generalize the algorithm to any number of prox calls per iteration, not only one, and propose a simple proof of linear convergence when the functions are smooth and strongly convex.