Fast Gradient Methods with Alignment for Symmetric Linear Systems without Using Cauchy Step

TL;DR

This paper introduces three new fast gradient methods with alignment for symmetric linear systems that do not rely on the Cauchy step, offering competitive performance and stability improvements over existing methods.

Contribution

It reveals spectral properties of optimal steps and proposes novel alignment gradient methods without Cauchy steps, enhancing practical efficiency.

Findings

New methods are competitive with Cauchy-based methods.

Alignment methods show increased stability in numerical experiments.

Advantages over Krylov subspace methods in certain scenarios.

Abstract

The performance of gradient methods has been considerably improved by the introduction of delayed parameters. After two and a half decades, the revealing of second-order information has recently given rise to the Cauchy-based methods with alignment, which reduce asymptotically the search spaces in smaller and smaller dimensions. They are generally considered as the state of the art of gradient methods. This paper reveals the spectral properties of minimal gradient and asymptotically optimal steps, and then suggests three fast methods with alignment without using the Cauchy step. The convergence results are provided, and numerical experiments show that the new methods provide competitive and more stable alternatives to the classical Cauchy-based methods. In particular, alignment gradient methods present advantages over the Krylov subspace methods in some situations, which makes them attractive in practice.