Conditions for linear convergence of the gradient method for non-convex optimization

TL;DR

This paper establishes that the Polyak-Lojasiewicz inequality is both necessary and sufficient for the linear convergence of the gradient method with fixed step sizes in certain non-convex smooth optimization problems.

Contribution

It provides a new linear convergence rate for the gradient method under the PL inequality and clarifies its role as a necessary and sufficient condition.

Findings

PL inequality is necessary and sufficient for linear convergence

Identifies classes of functions with linear convergence

Explores relationships between these classes and PL inequality

Abstract

In this paper, we derive a new linear convergence rate for the gradient method with fixed step lengths for non-convex smooth optimization problems satisfying the Polyak-Lojasiewicz (PL) inequality. We establish that the PL inequality is a necessary and sufficient condition for linear convergence to the optimal value for this class of problems. We list some related classes of functions for which the gradient method may enjoy linear convergence rate. Moreover, we investigate their relationship with the PL inequality.