Nonconvex Proximal Incremental Aggregated Gradient Method with Linear Convergence

TL;DR

This paper proves that the proximal incremental aggregated gradient (PIAG) algorithm achieves linear convergence in nonconvex optimization problems under certain conditions, extending its applicability beyond convex functions.

Contribution

The paper establishes linear convergence of the PIAG method for nonconvex functions using an error bound condition, a novel extension of prior convex-focused results.

Findings

Global convergence to stationary points

Linear convergence of objective and iterates under stepsize threshold

Applicability to nonconvex L-smooth functions

Abstract

In this paper, we study the proximal incremental aggregated gradient(PIAG) algorithm for minimizing the sum of L-smooth nonconvex component functions and a proper closed convex function. By exploiting the L-smooth property and with the help of an error bound condition, we can show that the PIAG method still enjoys some nice linear convergence properties even for nonconvex minimization. To illustrate this, we first demonstrate that the generated sequence globally converges to the stationary point set. Then, there exists a threshold such that the objective function value sequence and the iterate point sequence are R-linearly convergent when the stepsize is chosen below this threshold.