Global convergence and asymptotic optimality of the heavy ball method for a class of non-convex optimization problems

TL;DR

This paper analyzes the heavy ball optimization method, establishing conditions for its global convergence and demonstrating its superior convergence rate over the triple momentum method for certain non-convex problems.

Contribution

It characterizes parameter settings that ensure global convergence of the heavy ball method for non-convex functions with sector-bounded gradients and compares its performance to the triple momentum method.

Findings

Heavy ball method achieves global convergence under specific parameters.

The convergence factor of the heavy ball method is better than that of the triple momentum method.

The results apply to a class of non-convex problems with sector-bounded gradients.

Abstract

In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient. We characterize the parameters that render the method globally convergent and yield the best $R$-convergence factor. We show that for this family of functions, this convergence factor is superior to the factor obtained from the triple momentum method.