Solution of a NxN System of Linear algebraic Equations: 1 -- The Steepest Descent Method Revisited

TL;DR

This paper explores innovative modifications to the steepest descent method for solving NxN linear systems, aiming to enhance convergence speed through randomization and matrix transformations, as a foundation for future practical algorithms.

Contribution

It introduces novel approaches involving random movement and matrix transformations to improve the steepest descent method's convergence rate.

Findings

Significant performance improvements demonstrated in computational experiments.

Potential for developing practical algorithms based on these modifications.

Sets groundwork for future research in efficient linear system solvers.

Abstract

This is the first in a series of papers which deal with the development of novel methods for solving a system of linear algebraic equations with a time complexity lower than existing algorithms. The NxN system of linear equations, Ax = b, is often solved iteratively by minimizing the corresponding quadratic form using well known optimization techniques. The simplest of these is the steepest descent method, whose approach to the solution is usually quite rapid at the beginning but slows down drastically after a few iterations. This paper investigates possible approaches which can reduce or avoid this slowing down. The two approaches used here involve random movement of the point between iterations, and possible matrix transformations between iterations. This paper reports the results of computational experiments and shows the remarkable improvement in performance of the steepest descent method that is possible. The approaches described here do not give a practical algorithm right away. However, they set the stage for developing practical algorithms based on SD alone, which will be presented in later publications.