Exploring an Alternative Line-Search Method for Lagrange-Newton Optimization

TL;DR

This paper introduces a novel line-search method for Lagrange-Newton optimization that uses divergence of Newton step vectors, aiming to improve step damping near stationary points with complex valley structures.

Contribution

It proposes a new line-search criterion based on Newton step divergence and a zigzag traversal strategy for better convergence in constrained optimization.

Findings

The new method successfully finds stationary points from most starting points.

It demonstrates the ability to damp step lengths in tight valleys.

The utility of the method remains uncertain due to mixed results.

Abstract

In the Lagrange-Newton method, where Newton's method is applied to a Lagrangian function that includes equality constraints, all stationary points are saddle points. It is therefore not possible to use a line-search method based on the value of the objective function; instead, the line search can operate on merit functions. In this report, we explore an alternative line-search method which is applicable to this case; it particulary addresses the damping of the step length in tight valleys. We propose a line-search criterion based on the divergence of the field of Newton step vectors. The visualization of the criterion for two-dimensional test functions reveals a network of ravines with flat bottom at the zero points of the criterion. The ravines are typically connected to stationary points. To traverse this ravine network in order to approach a stationary point, a zigzag strategy is devised. Numerical experiments demonstrate that the novel line-search strategy succeeds from most starting points in all test functions, but only exhibits the desired damping of the step length in some situations. At the present stage it is therefore difficult to appraise the utility of this contribution.