A fresh look at nonsmooth Levenberg--Marquardt methods with applications to bilevel optimization

TL;DR

This paper introduces a nonsmooth Levenberg--Marquardt method that does not require local Lipschitzness, using Newton-differentiability, and applies it to bilevel optimization problems with promising numerical results.

Contribution

It proposes a novel nonsmooth Levenberg--Marquardt algorithm based on Newton-differentiability, extending solution techniques for nonsmooth systems and bilevel optimization.

Findings

The method achieves fast local convergence under new conditions.

Globalized solution methods are constructed for complementarity systems.

Numerical experiments show the effectiveness of the approach, including penalty parameter treatment.

Abstract

In this paper, we revisit the classical problem of solving over-determined systems of nonsmooth equations numerically. We suggest a nonsmooth Levenberg--Marquardt method for its solution which, in contrast to the existing literature, does not require local Lipschitzness of the data functions. This is possible when using Newton-differentiability instead of semismoothness as the underlying tool of generalized differentiation. Conditions for fast local convergence of the method are given. Afterwards, in the context of over-determined mixed nonlinear complementarity systems, our findings are applied, and globalized solution methods, based on a residual induced by the maximum and the Fischer--Burmeister function, respectively, are constructed. The assumptions for fast local convergence are worked out and compared. Finally, these methods are applied for the numerical solution of bilevel optimization problems. We recall the derivation of a stationarity condition taking the shape of an over-determined mixed nonlinear complementarity system involving a penalty parameter, formulate assumptions for local fast convergence of our solution methods explicitly, and present results of numerical experiments. Particularly, we investigate whether the treatment of the appearing penalty parameter as an additional variable is beneficial or not.