A Generalized Alternating Anderson Acceleration Method

TL;DR

This paper introduces a flexible generalized alternating Anderson acceleration scheme that combines fixed-point iterations with Anderson acceleration to improve convergence in solving linear and nonlinear problems, demonstrating superior efficiency over existing methods.

Contribution

It proposes a novel generalized alternating scheme with convergence analysis and broad applicability to various iterative methods for PDEs and optimization problems.

Findings

More efficient than existing Anderson acceleration methods

Proven convergence under certain conditions for linear problems

Effective acceleration for multiple iterative algorithms

Abstract

In this work, we propose a generalized alternating Anderson acceleration method, a periodic scheme composed of $t$ fixed-point iteration steps, interleaved with $s$ steps of Anderson acceleration with window size $m$, to solve linear and nonlinear problems. This allows flexibility to use different combinations of fixed-point iteration and Anderson iteration. We present a convergence analysis of the proposed scheme for accelerating the Richardson iteration in the linear case, with a focus on specific parameter choices of interest. Specifically, we prove convergence of the proposed method under contractive fixed-point iteration and provide a sufficient condition for convergence when the Richardson iteration matrix is diagonalizable and noncontractive. To demonstrate the broader applicability of our proposed method, we use it to accelerate Jacobi iteration, Picard iteration, gradient descent, and the alternating direction method of multipliers in solving partial differential equations and nonlinear, nonsmooth optimization problems. The numerical results illustrate that the proposed scheme is more efficient than the existing windowed Anderson acceleration and alternating Anderson ($s=1$) in terms of iteration number and CPU time for careful choice of parameters $m, s, t$.