A smoothing Anderson acceleration algorithm for nonsmooth fixed point problem with linear convergence

TL;DR

This paper introduces a Smoothing Anderson acceleration algorithm with linear convergence for nonsmooth fixed point problems, demonstrating theoretical guarantees and practical improvements over existing methods.

Contribution

The paper proposes a novel Smoothing Anderson(m) algorithm with adaptive smoothing for nonsmooth fixed point problems, establishing its linear convergence and superior performance.

Findings

Proves r-linear convergence of the proposed algorithm.

Shows q-linear convergence of related algorithms.

Demonstrates improved numerical performance in practical applications.

Abstract

In this paper, we consider the Anderson acceleration method for solving the contractive fixed point problem, which is nonsmooth in general. We define a class of smoothing functions for the original nonsmooth fixed point mapping, which can be easily formulated for many cases (see section3). Then, taking advantage of the Anderson acceleration method, we proposed a Smoothing Anderson(m) algorithm, in which we utilized a smoothing function of the original nonsmooth fixed point mapping and update the smoothing parameter adaptively. In theory, we first demonstrate the r-linear convergence of the proposed Smoothing Anderson(m) algorithm for solving the considered nonsmooth contractive fixed point problem with r-factor no larger than c, where c is the contractive factor of the fixed point mapping. Second, we establish that both of the Smoothing Anderson(1) and the Smoothing EDIIS(1) algorithms are q-linear convergent with q-factor no larger than c. Finally, we present three numerical examples with practical applications from elastic net regression, free boundary problems for infinite journal bearings and non-negative least squares problem to illustrate the better performance of the proposed Smoothing Anderson(m) algorithm comparing with some popular methods.