Stochastic Gradient Descent for Nonlinear Inverse Problems in Banach Spaces

TL;DR

This paper analyzes the use of stochastic gradient descent for solving nonlinear inverse problems in Banach spaces, proving convergence and regularization properties, with numerical validation on tomography applications.

Contribution

It extends SGD analysis to nonlinear inverse problems in Banach spaces, establishing convergence, regularization, and rates under general assumptions.

Findings

Almost sure convergence to minimum distance solution.

Regularization in expectation with an a priori stopping rule.

Convergence rates under stability assumptions.

Abstract

Stochastic gradient descent (SGD) and its variants are widely used and highly effective optimization methods in machine learning, especially for neural network training. By using a single datum or a small subset of the data, selected randomly at each iteration, SGD scales well to problem size and has been shown to be effective for solving large-scale inverse problems. In this work, we investigate SGD for solving nonlinear inverse problems in Banach spaces through the lens of iterative regularization. Under general assumptions, we prove almost sure convergence of the iterates to the minimum distance solution and show the regularizing property in expectation under an a priori stopping rule. Further, we establish convergence rates under the conditional stability assumptions for both exact and noisy data. Numerical experiments on Schlieren tomography and electrical impedance tomography are presented to show distinct features of the method.