On a posteriori stopping rules of adaptive stochastic heavy ball method for ill-posed problems

TL;DR

This paper introduces an adaptive stochastic heavy ball method with an { m a posteriori} stopping rule for efficiently solving large-scale ill-posed inverse problems, combining momentum, adaptive step sizes, and convex penalties.

Contribution

It develops a novel stochastic heavy ball algorithm with an adaptive stopping rule inspired by the discrepancy principle, improving efficiency for ill-posed problems.

Findings

Method converges almost surely and in expectation under certain conditions.

Numerical experiments show the method's efficiency and potential for large-scale problems.

Adaptive strategy enhances convergence speed and computational practicality.

Abstract

In this paper we develop a stochastic heavy ball method for solving ill-posed inverse problems. The method updates the iterate using only a randomly selected equation at each iteration step while incorporating a momentum term into the process. To facilitate fast convergence, we propose an adaptive strategy for selecting the step size and the momentum coefficient. Inspired by the spirit of the discrepancy principle, we introduce an {\it a posteriori} stopping rule for our adaptive stochastic heavy ball method. This rule avoids the need to compute residuals of all equations in the system at every iteration or at fixed frequency intervals, thereby enhancing computational efficiency and practicality. Additionally, convex penalty functions are employed to capture the specific features of the desired solutions. Under suitable conditions, we establish almost sure convergence as well as convergence in expectation. Extensive numerical experiments are conducted to evaluate the performance of the proposed method, demonstrating its efficiency and promising potential for solving large-scale ill-posed problems.