Safeguarded Anderson acceleration for parametric nonexpansive operators

TL;DR

This paper introduces a safeguarding scheme for Anderson acceleration to enhance stability and performance in first-order optimization methods, demonstrated across various quadratic and semidefinite programming problems.

Contribution

It proposes a novel safeguarding scheme combining non-expansiveness, conditioning, and restarts, integrated into Anderson acceleration for improved optimization stability.

Findings

Effective on seven different problem types

Successfully tested on over 500 problems

Implemented in open-source COSMO solver

Abstract

This paper describes the design of a safeguarding scheme for Anderson acceleration to improve its practical performance and stability when used for first-order optimisation methods. We show how the combination of a non-expansiveness condition, conditioning constraints, and memory restarts integrate well with solver algorithms that can be represented as fixed point operators with dynamically varying parameters. The performance of the scheme is demonstrated on seven different QP and SDP problem types, including more than 500 problems. The safeguarded Anderson acceleration scheme proposed in this paper is implemented in the open-source ADMM-based conic solver COSMO.