Acceleration methods for fixed point iterations

TL;DR

This paper reviews various acceleration techniques for fixed point iterations, highlighting methods like Anderson Acceleration that improve convergence speed in scientific computing.

Contribution

It provides a comprehensive overview of fixed point acceleration methods, emphasizing Anderson Acceleration and related techniques, and discusses their applications and effectiveness.

Findings

Anderson Acceleration significantly speeds up fixed point iterations.

Extrapolation methods like Aitken's process are simple but limited.

Accelerating fixed point methods enhances performance in physics simulations.

Abstract

A pervasive approach in scientific computing is to express the solution to a given problem as the limit of a sequence of vectors or other mathematical objects. In many situations these sequences are generated by slowly converging iterative procedures and this led practitioners to seek faster alternatives to reach the limit. ``Acceleration techniques'' comprise a broad array of methods specifically designed with this goal in mind. They started as a means of improving the convergence of general scalar sequences by various forms of ``extrapolation to the limit'', i.e., by extrapolating the most recent iterates to the limit via linear combinations. Extrapolation methods of this type, the best known example of which is Aitken's Delta-squared process, require only the sequence of vectors as input. However, limiting methods to only use the iterates is too restrictive. Accelerating sequences generated by fixed-point iterations by utilizing both the iterates and the fixed-point mapping itself has proven highly successful across various areas of physics. A notable example of these Fixed-Point accelerators (FP-Accelerators) is a method developed by D. Anderson in 1965 and now widely known as Anderson Acceleration (AA). Furthermore, Quasi-Newton and Inexact Newton methods can also be placed in this category as well. This paper presents an overview of these methods -- with an emphasis on those, such as AA, that are geared toward accelerating fixed point iterations.