A Non-Monotone Line-Search Method for Minimizing Functions with Spurious Local Minima

TL;DR

This paper introduces a non-monotone line-search optimization method that effectively escapes spurious local minima in smooth functions, with proven complexity bounds and superior performance in numerical tests.

Contribution

It presents a novel non-monotone line-search technique based on a relaxed Armijo condition, improving optimization in functions with many local minima.

Findings

Outperforms existing non-monotone methods on functions with spurious minima

Provides worst-case complexity estimates for the proposed method

Numerical results demonstrate significant efficiency gains

Abstract

In this paper, we propose a new non-monotone line-search method for smooth unconstrained optimization problems with objective functions that have many non-global local minimizers. The method is based on a relaxed Armijo condition that allows a controllable increase in the objective function between consecutive iterations. This property helps the iterates escape from nearby local minimizers in the early iterations. For objective functions with Lipschitz continuous gradients, we derive worst-case complexity estimates on the number of iterations needed for the method to find approximate stationary points. Numerical results are presented, showing that the new method can significantly outperform other non-monotone methods on functions with spurious local minima.