Average block nonlinear Kaczmarz methods with adaptive momentum for nonlinear systems of equations

TL;DR

This paper introduces an adaptive momentum-accelerated block nonlinear Kaczmarz method for large-scale nonlinear systems, demonstrating improved efficiency and convergence properties through theoretical analysis and numerical experiments.

Contribution

It develops a new adaptive momentum strategy for nonlinear Kaczmarz methods, enhancing computational efficiency and providing rigorous convergence analysis.

Findings

ABNKAm requires minimal inner-product computations per iteration.

The method converges exponentially to the nearest solution.

Numerical results show superior performance over existing methods.

Abstract

The Kaczmarz method is widely recognized as an efficient iterative algorithm for solving large-scale linear systems, owing to its simplicity and low memory requirements. However, the development of its nonlinear extensions for solving large-scale nonlinear systems has seen limited progress. In this work, we introduce a new family of momentum-accelerated averaging block nonlinear Kaczmarz methods tailored for large-scale nonlinear systems and ill-posed problems. Our contributions are twofold: (1) We develop an adaptive strategy for selecting step sizes and momentum coefficients, leading to a new average block nonlinear Kaczmarz method with adaptive momentum (ABNKAm). This algorithm achieves high computational efficiency by requiring only minimal inner-product computations per iteration, which significantly reduces both arithmetic complexity and memory usage. (2) We establish rigorous convergence of the ABNKAm under mild assumptions, proving that the method converges exponentially to the unique solution nearest to the initial point. Moreover, under suitable conditions, we provide a theoretical justification of acceleration of the proposed ABNKAm with momentum. Extensive numerical experiments demonstrate that ABNKAm outperforms existing nonlinear Kaczmarz variants in terms of both iteration count and computational time, with particularly notable gains in large-scale problems.