Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification

TL;DR

This paper proves local convergence of Anderson acceleration for nonsmooth optimization algorithms that identify active manifolds, supported by numerical experiments demonstrating robust performance.

Contribution

It establishes local R-linear convergence of Anderson acceleration in nonsmooth problems with active manifold identification, covering various algorithms.

Findings

Proves local R-linear convergence under active manifold assumptions

Includes diverse algorithms like proximal methods and ADMM

Numerical experiments show robust performance

Abstract

Anderson acceleration is an effective technique for enhancing the efficiency of fixed-point iterations; however, analyzing its convergence in nonsmooth settings presents significant challenges. In this paper, we investigate a class of nonsmooth optimization algorithms characterized by the active manifold identification property. This class includes a diverse array of methods such as the proximal point method, proximal gradient method, proximal linear method, proximal coordinate descent method, Douglas-Rachford splitting (or the alternating direction method of multipliers), and the iteratively reweighted $\ell_1$ method, among others. Under the assumption that the optimization problem possesses an active manifold at a stationary point, we establish a local R-linear convergence rate for the Anderson-accelerated algorithm. Our extensive numerical experiments further highlight the robust performance of the proposed Anderson-accelerated methods.