On growth error bound conditions with an application to heavy ball method

TL;DR

This paper explores growth error bound conditions, linking them to Kurdyka-Łojasiewicz conditions, and applies these insights to improve the heavy ball optimization method with an adaptive momentum strategy, demonstrating accelerated convergence.

Contribution

It provides an elementary proof connecting Kurdyka-Łojasiewicz and growth error bounds, extends results to nonconvex functions, and proposes an adaptive heavy ball method with proven convergence rates.

Findings

Growth error bounds hold for definable functions in o-minimal structures.

The adaptive heavy ball method accelerates convergence over standard gradient methods.

Numerical results confirm the effectiveness of the proposed acceleration strategy.

Abstract

In this paper, we investigate the growth error bound condition. By using the proximal point algorithm, we first provide a more accessible and elementary proof of the fact that Kurdyka-{\L}ojasiewicz conditions imply growth error bound conditions for convex functions which has been established before via a subgradient flow. We then extend the result for nonconvex functions. Furthermore we show that every definable function in an o-minimal structure must satisfy a growth error bound condition. Finally, as an application, we consider the heavy ball method for solving convex optimization problems and propose an adaptive strategy for selecting the momentum coefficient. Under growth error bound conditions, we derive convergence rates of the proposed method. A numerical experiment is conducted to demonstrate its acceleration effect over the gradient method.