A derivative-free regularization algorithm for equality constrained nonlinear least squares problems

TL;DR

This paper introduces a derivative-free regularization algorithm for equality constrained nonlinear least squares problems, effectively handling scenarios where Jacobian matrices are unavailable or costly to compute, by using orthogonal spherical smoothing.

Contribution

It proposes a novel derivative-free regularization method that combines augmented Lagrangian and Levenberg-Marquardt techniques for constrained nonlinear least squares.

Findings

Algorithm converges to an approximate KKT point with high probability

Method effectively handles problems without explicit Jacobian computations

Convergence to stationary points of constraint violation is established

Abstract

In this paper, we study the equality constrained nonlinear least squares problem, where the Jacobian matrices of the objective function and constraints are unavailable or expensive to compute. We approximate the Jacobian matrices via orthogonal spherical smoothing and propose a derivative-free regularization algorithm for solving the problem. At each iteration, a regularized augmented Lagrangian subproblem is solved to obtain a Newton-like step. If a sufficient decrease in the merit function of the approximate KKT system is achieved, the step is accepted, otherwise a derivative-free LM algorithm is applied to get another step to satisfy the sufficient decrease condition. It is shown that the algorithm either finds an approximate KKT point with arbitrary high probability or converges to a stationary point of constraints violation almost surely.