A Convergence Study for Reduced Rank Extrapolation on Nonlinear Systems

TL;DR

This paper investigates the convergence behavior of Reduced Rank Extrapolation (RRE) when applied to nonlinear systems, extending previous linear analyses to nonlinear cases and exploring convergence in different modes.

Contribution

It provides a detailed analysis of RRE convergence for nonlinear functions, including fixed and cycling modes, which was not thoroughly studied before.

Findings

Convergence of RRE with nonlinear functions when n approaches infinity with fixed k.

Analysis of RRE convergence in cycling modes for nonlinear systems.

Extension of linear RRE convergence results to nonlinear cases.

Abstract

Reduced Rank Extrapolation (RRE) is a polynomial type method used to accelerate the convergence of sequences of vectors $\{\boldsymbol{x}_m\}$. It is applied successfully in different disciplines of science and engineering in the solution of large and sparse systems of linear and nonlinear equations of very large dimension. If $\boldsymbol{s}$ is the solution to the system of equations $\boldsymbol{x}=\boldsymbol{f}(\boldsymbol{x})$, first, a vector sequence $\{\boldsymbol{x}_m\}$ is generated via the fixed-point iterative scheme $\boldsymbol{x}_{m+1}=\boldsymbol{f}(\boldsymbol{x}_m)$, $m=0,1,\ldots,$ and next, RRE is applied to this sequence to accelerate its convergence. RRE produces approximations $\boldsymbol{s}_{n,k}$ to $\boldsymbol{s}$ that are of the form $\boldsymbol{s}_{n,k}=\sum^k_{i=0}\gamma_i\boldsymbol{x}_{n+i}$ for some scalars $\gamma_i$ depending (nonlinearly) on $\boldsymbol{x}_n, \boldsymbol{x}_{n+1},\ldots,\boldsymbol{x}_{n+k+1}$ and satisfying $\sum^k_{i=0}\gamma_i=1$. The convergence properties of RRE when applied in conjunction with linear $\boldsymbol{f}(\boldsymbol{x})$ have been analyzed in different publications. In this work, we discuss the convergence of the $\boldsymbol{s}_{n,k}$ obtained from RRE with nonlinear $\boldsymbol{f}(\boldsymbol{x})$ (i)\,when $n\to\infty$ with fixed $k$, and (ii)\,in two so-called {\em cycling} modes.