A characterization of the behavior of the Anderson acceleration on linear problems

TL;DR

This paper provides a complete characterization of Anderson acceleration's behavior on linear problems, showing conditions for convergence to the correct solution or stagnation, and relating it to GMRES.

Contribution

It offers a theoretical analysis of Anderson acceleration on linear problems, detailing when it converges correctly or stagnates, and its relation to GMRES.

Findings

If Anderson acceleration does not stagnate up to the residual grade, it converges to the exact solution.

Stagnation leads to convergence to an incorrect solution.

Anderson acceleration is essentially equivalent to GMRES up to the stagnation point.

Abstract

We give a complete characterization of the behavior of the Anderson acceleration (with arbitrary nonzero mixing parameters) on linear problems. Let n be the grade of the residual at the starting point with respect to the matrix defining the linear problem. We show that if Anderson acceleration does not stagnate (that is, produces different iterates) up to n, then the sequence of its iterates converges to the exact solution of the linear problem. Otherwise, the Anderson acceleration converges to the wrong solution. Anderson acceleration and of GMRES are essentially equivalent up to the index where the iterates of Anderson acceleration begin to stagnate. This result holds also for an optimized version of Anderson acceleration, where at each step the mixing parameter is chosen so that it minimizes the residual of the current iterate.