Convergence Analysis for Restarted Anderson Mixing and Beyond

TL;DR

This paper provides a unified convergence analysis for restarted Anderson Mixing methods, demonstrating their local convergence improvement and proposing adaptive strategies based on Jacobian spectrum estimation.

Contribution

It introduces a unified convergence analysis for restarted AM, proposes an adaptive mixing strategy, and develops memory-efficient forms for symmetric Jacobians.

Findings

Restarted Type-II AM can locally improve convergence rate.

Adaptive mixing strategy based on Jacobian spectrum estimation.

Memory-efficient forms for symmetric Jacobian cases.

Abstract

Anderson mixing (AM) is a classical method that can accelerate fixed-point iterations by exploring historical information. Despite the successful application of AM in scientific computing, the theoretical properties of AM are still under exploration. In this paper, we study the restarted version of the Type-I and Type-II AM methods, i.e., restarted AM. With a multi-step analysis, we give a unified convergence analysis for the two types of restarted AM and justify that the restarted Type-II AM can locally improve the convergence rate of the fixed-point iteration. Furthermore, we propose an adaptive mixing strategy by estimating the spectrum of the Jacobian matrix. If the Jacobian matrix is symmetric, we develop the short-term recurrence forms of restarted AM to reduce the memory cost. Finally, experimental results on various problems validate our theoretical findings.