Stabilization of the Gradient Method for Solving Linear Algebraic Systems -- A Method Related to the Normal Equation

TL;DR

This paper introduces a simple, robust stabilization technique for the gradient method that guarantees convergence for any nonsingular matrix without requiring matrix structure or stepsize tuning.

Contribution

It proposes a new stabilized gradient method that converges globally for all nonsingular matrices, regardless of their structure or conditioning.

Findings

Successfully converges on small and large scale systems

Performs well on structured and unstructured matrices

Handles well-conditioned and ill-conditioned matrices

Abstract

Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of the stepsize defining the new iteration. We propose here a simple and robust stabilization of the gradient method, which no longer assumes a structure on the matrix (neither symmetric, nor positive definite) to converge, and which no longer requires an approach on the choice of the stepsize. We establish the global convergence of the proposed stabilized algorithm under the only assumption of nonsingular matrix. Several numerical examples illustrating its performances are presented, where we have tested small and large scale linear systems, with and not structured matrices, and with well and ill conditioned matrices.