A heavy-ball type curve search method for smooth convexly constrained optimization

TL;DR

This paper introduces a novel heavy-ball-type optimization method for smooth convex problems with constraints, combining momentum and curvilinear search to ensure convergence and feasibility, demonstrated to be effective through numerical experiments.

Contribution

It extends the heavy-ball momentum technique to constrained optimization using a curve-search framework, ensuring convergence and feasibility in a novel way.

Findings

Method is globally convergent to stationary points.

Algorithm is robust and competitive on benchmark problems.

Incorporates adaptive momentum and spectral steplength strategies.

Abstract

This paper addresses smooth convexly constrained optimization problems where the Euclidean projection onto the feasible set is computationally tractable. Although momentum techniques like Polyak's heavy-ball are known for accelerating optimization algorithms, their use in constrained settings remains limited due to challenges in preserving feasibility and ensuring convergence. We thus propose a heavy-ball-type method that extends to the constrained case a recently introduced curve-search globalization framework. The method attempts a momentum update and performs a curvilinear search to enforce an Armijo-type descent condition: when the momentum step is infeasible or unacceptable, the algorithm smoothly reverts to a feasible descent direction. We prove that the algorithm is well-defined and globally convergent to stationary points; the derivation of these results is nontrivial due to the use of a heavy-ball type direction in a constrained setting, where it may generate infeasible iterates. We discuss the incorporation of further mechanisms into the algorithm, including non-monotone curve search, spectral steplength selection and an adaptive momentum strategy. Numerical experiments on benchmark problems show the method is robust and competitive with the state-of-the-art.