Anderson acceleration for contractive and noncontractive operators

TL;DR

This paper provides a new theoretical analysis of Anderson acceleration for both contractive and noncontractive operators, offering sharper residual bounds and practical guidance for dynamic algorithmic depth selection.

Contribution

It introduces a one-step residual analysis that improves understanding of Anderson acceleration, especially in noncontractive settings, with a safeguarding strategy and numerical validation.

Findings

Residual bounds depend on higher order terms and optimization success.

Bounds are sharper in the contractive setting, improving previous results.

Numerical tests demonstrate effectiveness on complex PDEs.

Abstract

A one-step analysis of Anderson acceleration with general algorithmic depths is presented. The resulting residual bounds within both contractive and noncontractive settings reveal the balance between the contributions from the higher and lower order terms, which are both dependent on the success of the optimization problem solved at each step of the algorithm. The new residual bounds show the additional terms introduced by the extrapolation produce terms that are of a higher order than was previously understood. In the contractive setting, these bounds sharpen previous convergence and acceleration results. The bounds rely on sufficient linear independence of the differences between consecutive residuals, rather than assumptions on the boundedness of the optimization coefficients, allowing the introduction of a theoretically sound safeguarding strategy. Several numerical tests illustrate the analysis primarily in the noncontractive setting, and demonstrate the use of the method, the safeguarding strategy, and theory-based guidance on dynamic selection of the algorithmic depth, on a p-Laplace equation, a nonlinear Helmholtz equation, and the steady Navier-Stokes equations with high Reynolds number in three spatial dimensions.