On the global complexity of a derivative-free Levenberg-Marquardt algorithm via orthogonal spherical smoothing

TL;DR

This paper introduces a derivative-free Levenberg-Marquardt algorithm that uses orthogonal spherical smoothing to approximate Jacobians, providing probabilistic complexity bounds for solving nonlinear least squares problems.

Contribution

It presents a novel derivative-free approach with probabilistic complexity analysis for Levenberg-Marquardt algorithms using orthogonal spherical smoothing.

Findings

Gradient models are probabilistically first-order accurate.

The algorithm has a high probability complexity bound.

The method effectively approximates Jacobians without derivatives.

Abstract

In this paper, we propose a derivative-free Levenberg-Marquardt algorithm for nonlinear least squares problems, where the Jacobian matrices are approximated via orthogonal spherical smoothing. It is shown that the gradient models which use the approximate Jacobian matrices are probabilistically first-order accurate, and the high probability complexity bound of the algorithm is also given.