Richardson Approach or Direct Methods? What to Apply in the Ill-Conditioned Least Squares Problem

TL;DR

This paper compares direct methods and Richardson iteration for solving ill-conditioned linear systems, demonstrating that Richardson iteration with a simple preconditioner yields more accurate and stable results without peaking phenomena.

Contribution

The study shows that Richardson iteration with a simple preconditioner outperforms direct methods in accuracy and robustness for ill-conditioned problems.

Findings

Direct methods produce inaccurate results with ill-conditioned matrices.

Richardson iteration avoids peaking phenomena and provides stable solutions.

A simple preconditioner based on maximum row sum norm is more robust than eigenvalue-based preconditioning.

Abstract

This report shows on real data that the direct methods such as LDL decomposition and Gaussian elimination for solving linear systems with ill-conditioned matrices provide inaccurate results due to divisions by very small numbers, which in turn results in peaking phenomena and large estimation errors. Richardson iteration provides accurate results without peaking phenomena since division by small numbers is absent in the Richardson approach. In addition, two preconditioners are considered and compared in the Richardson iteration: 1) the simplest and robust preconditioner based on the maximum row sum matrix norm and 2) the optimal one based on calculation of the eigenvalues. It is shown that the simplest preconditioner is more robust for ill-conditioned case and therefore it is recommended for many applications.