Fast Krasnosel'skii-Mann algorithm with a convergence rate of the fixed point iteration of $o\left(\frac{1}{k}\right)$

TL;DR

This paper introduces a Fast Krasnosel'skii-Mann algorithm enhanced with Nesterov's momentum, achieving a convergence rate of o(1/k) for fixed point residuals while maintaining weak convergence, and demonstrates its effectiveness through numerical experiments.

Contribution

The paper develops a novel accelerated KM algorithm with Nesterov's momentum that improves convergence rate without sacrificing convergence guarantees.

Findings

Achieves a convergence rate of o(1/k) for fixed point residuals.

Maintains weak convergence of iterates to a fixed point.

Numerical experiments show superior performance over existing schemes.

Abstract

The Krasnosel'skii-Mann (KM) algorithm is the most fundamental iterative scheme designed to find a fixed point of an averaged operator in the framework of a real Hilbert space, since it lies at the heart of various numerical algorithms for solving monotone inclusions and convex optimization problems. We enhance the Krasnosel'skii-Mann algorithm with Nesterov's momentum updates and show that the resulting numerical method exhibits a convergence rate for the fixed point residual of $o(1/k)$ while preserving the weak convergence of the iterates to a fixed point of the operator. Numerical experiments illustrate the superiority of the resulting so-called Fast KM algorithm over various fixed point iterative schemes, and also its oscillatory behavior, which is a specific of Nesterov's momentum optimization algorithms.