A link between the steepest descent method and fixed-point iterations

TL;DR

This paper establishes a connection between the steepest descent method and fixed-point iterations for solving unconstrained minimization problems, revealing insights and rediscovering known algorithms through this link, supported by numerical experiments.

Contribution

It introduces a novel theoretical link between steepest descent and fixed-point iterations, providing a new perspective on optimization algorithms.

Findings

The link between steepest descent and fixed-point iterations is established.

The preconditioned nonlinear conjugate gradient method is rediscovered.

Numerical experiments illustrate the benefits of this connection.

Abstract

We will make a link between the steepest descent method for an unconstrained minimisation problem and fixed-point iterations for its Euler-Lagrange equation. In this context, we shall rediscover the preconditioned nonlinear conjugate gradient method for the discretised problem. The benefit of the link between the two methods will be illustrated by a numerical experiment.