A Stochastic Iteratively Regularized Gauss-Newton Method

TL;DR

This paper introduces a stochastic version of the iteratively regularized Gauss-Newton method for inverse problems, using mini-batching and randomized sketching to improve computational efficiency while maintaining accuracy.

Contribution

It develops the stochastic iteratively regularized Gauss-Newton method (SIRGNM) with theoretical analysis and demonstrates its effectiveness through numerical experiments.

Findings

SIRGNM maintains accuracy comparable to deterministic methods.

SIRGNM reduces computational time in numerical experiments.

Theoretical convergence analysis supports the method's reliability.

Abstract

This work focuses on developing and motivating a stochastic version of a wellknown inverse problem methodology. Specifically, we consider the iteratively regularized Gauss-Newton method, originally proposed by Bakushinskii for infinite-dimensional problems. Recent work have extended this method to handle sequential observations, rather than a single instance of the data, demonstrating notable improvements in reconstruction accuracy. In this paper, we further extend these methods to a stochastic framework through mini-batching, introducing a new algorithm, the stochastic iteratively regularized Gauss-Newton method (SIRGNM). Our algorithm is designed through the use randomized sketching. We provide an analysis for the SIRGNM, which includes a preliminary error decomposition and a convergence analysis, related to the residuals. We provide numerical experiments on a 2D elliptic PDE example. This illustrates the effectiveness of the SIRGNM, through maintaining a similar level of accuracy while reducing on the computational time.