A Jacobian Free Deterministic Method for Solving Inverse Problems

TL;DR

This paper introduces a Jacobian-free, derivative-free numerical method for solving inverse problems by optimizing model parameters through a non-linear least squares approach, suitable for complex systems with nested objectives.

Contribution

The method eliminates the need for explicit derivative calculations, using Broyden's updates and a line search to ensure convergence, enhancing robustness for complex inverse problems.

Findings

Effective for complex physical models

Insensitive to initial parameter estimates

Proven successful in practical inverse problems

Abstract

An effective numerical method is presented for optimizing model parameters that can be applied to any type of system of non-linear equations and any number of data-points, which does not require explicit formulation of the objective function or its partial derivatives. The numerics are reduced to solving a non-linear least squares problem, which uses the Levenberg-Marquardt algorithm and the Jacobian is approximated by applying rank-one updates using Broyden's method. An advantage of this methodology over conventional approaches is that the partial derivatives of the objective function do not have to be analytically calculated. For instance, there may be situations where one cannot formulate the partial derivatives, such as cases involving an objective function that itself contains a nested optimization problem. Moreover, a line search algorithm is also described that ensures that the Armijo conditions are satisfied and that convergence is assured, which makes the success of the approach insensitive to the initial estimates of the model parameters. The foregoing numerical methods are described with respect to the development of the Optima software to solve inverse problems, which are reduced to non-linear least squares problems. This computational approach has proven to be particularly useful at solving inverse problems of very complex physical models that cannot be optimized directly in a practical way.