From Halpern's Fixed-Point Iterations to Nesterov's Accelerated Interpretations for Root-Finding Problems

TL;DR

This paper establishes a Nesterov's accelerated interpretation of Halpern's fixed-point iteration for root-finding, deriving new convergence rates and applying them to monotone inclusion problems with practical numerical validation.

Contribution

It introduces a Nesterov's accelerated framework for fixed-point iterations, extending to methods requiring only monotonicity and Lipschitz continuity, and provides new convergence analyses.

Findings

Derived a new convergence rate for Halpern's fixed-point iteration.

Established equivalence between Halpern's scheme and Nesterov's acceleration.

Validated theoretical results with numerical experiments.

Abstract

We derive an equivalent form of Halpern's fixed-point iteration scheme for solving a co-coercive equation (also called a root-finding problem), which can be viewed as a Nesterov's accelerated interpretation. We show that one method is equivalent to another via a simple transformation, leading to a straightforward convergence proof for Nesterov's accelerated scheme. Alternatively, we directly establish convergence rates of Nesterov's accelerated variant, and as a consequence, we obtain a new convergence rate of Halpern's fixed-point iteration. Next, we apply our results to different methods to solve monotone inclusions, where our convergence guarantees are applied. Since the gradient/forward scheme requires the co-coerciveness of the underlying operator, we derive new Nesterov's accelerated variants for both recent extra-anchored gradient and past-extra anchored gradient methods in the literature. These variants alleviate the co-coerciveness condition by only assuming the monotonicity and Lipschitz continuity of the underlying operator. Interestingly, our new Nesterov's accelerated interpretation of the past-extra anchored gradient method involves two past-iterate correction terms. This formulation is expected to guide us developing new Nesterov's accelerated methods for minimax problems and their continuous views without co-coericiveness. We test our theoretical results on two numerical examples, where the actual convergence rates match well the theoretical ones up to a constant factor.