On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

TL;DR

This paper investigates the convergence and regularization properties of stochastic gradient descent when applied to large-scale nonlinear ill-posed inverse problems, demonstrating its effectiveness and scalability.

Contribution

It introduces a stochastic gradient descent approach for nonlinear inverse problems, proving regularization and convergence under specific conditions, extending classical methods.

Findings

Proves regularizing property of stochastic gradient descent for nonlinear inverse problems.

Establishes convergence rates under sourcewise and range invariance conditions.

Demonstrates scalability and potential for large-scale problem solving.

Abstract

In this work, we analyze the regularizing property of the stochastic gradient descent for the efficient numerical solution of a class of nonlinear ill-posed inverse problems in Hilbert spaces. At each step of the iteration, the method randomly chooses one equation from the nonlinear system to obtain an unbiased stochastic estimate of the gradient, and then performs a descent step with the estimated gradient. It is a randomized version of the classical Landweber method for nonlinear inverse problems, and it is highly scalable to the problem size and holds significant potentials for solving large-scale inverse problems. Under the canonical tangential cone condition, we prove the regularizing property for a priori stopping rules, and then establish the convergence rates under suitable sourcewise condition and range invariance condition.