Convergence Analysis of An Alternating Nonlinear GMRES on Linear Systems

TL;DR

This paper introduces an alternating nonlinear GMRES algorithm for linear systems, providing theoretical convergence analysis, showing equivalence to GMRES under certain conditions, and highlighting computational advantages and robustness improvements.

Contribution

The paper develops a new aNGMRES($m,p$) algorithm, analyzes its convergence, and establishes its equivalence to GMRES in specific cases, offering a cost-effective and robust alternative for linear system solutions.

Findings

aNGMRES($ ext{infinity},p$) is equivalent to GMRES at certain iterations

aNGMRES($m,m+1$) matches restarted GMRES at periodic intervals

aNGMRES accelerates Richardson iteration with improved robustness

Abstract

In this work, we develop an alternating nonlinear Generalized Minimum Residual (NGMRES) algorithm with depth $m$ and periodicity $p$, denoted by aNGMRES($m, p$), applied to linear systems. We provide a theoretical analysis to quantify by how much one-step NGMRES($m$) using Richardson iterations as initial guesses can improve the convergence speed of the underlying fixed-point iteration for diagonalizable and symmetric positive definite cases. Our theoretical analysis gives us a better understanding of which factors affect the convergence speed. Moreover, under certain conditions, we prove the periodic equivalence between the proposed aNGMRES applied to Richardson iteration and GMRES. Specifically, aNGMRES($\infty,p$) and full GMRES are identical at the iteration index $jp$. Therefore, aNGMRES($\infty,p$) can be regarded as an alternative to GMRES for solving linear systems. For finite $m$, the iterates of aNGMRES($m,m+1$) and restarted GMRES (GMRES($m+1$)) are the same at the end of each periodic interval of length $p$, i.e, at the iteration index $jp$. In Addition, we present a convergence analysis of aNGMRES when applied to accelerate Richardson iteration. The advantages of aNGMRES($m,p$) method are that there is no need to solve a least-squares problem at each iteration which can reduce the computational cost, and it can enhance the robustness against stagnations, which could occur for NGMRES($m$).