Anderson acceleration with adaptive relaxation for convergent fixed-point iterations

TL;DR

This paper introduces two adaptive relaxation strategies for Anderson acceleration, improving convergence speed and efficiency in fixed-point iterations, especially for linear contractions and nonlinear applications like PDEs and EM algorithms.

Contribution

The paper proposes novel adaptive relaxation methods for Anderson acceleration, enhancing convergence and computational efficiency in fixed-point problems.

Findings

Superiority over existing Anderson acceleration methods for linear contractions

Effective in nonlinear fixed-point applications including PDEs and EM algorithms

One strategy outperforms others in computation time across various tests

Abstract

Two adaptive relaxation strategies are proposed for Anderson acceleration. They are specifically designed for applications in which mappings converge to a fixed point. Their superiority over alternative Anderson acceleration is demonstrated for linear contraction mappings. Both strategies perform well in three nonlinear fixed-point applications that include partial differential equations and the EM algorithm. One strategy surpasses all other Anderson acceleration implementations tested in terms of computation time across various specifications, including composite Anderson acceleration.