On the iterative solution of systems of the form $A^T A x=A^Tb+c$

TL;DR

This paper introduces two iterative methods tailored for solving structured linear systems of the form $A^T A x = A^T b + c$, which incorporate structure-aware condition numbers to improve accuracy and robustness over standard approaches.

Contribution

The paper presents novel iterative algorithms for a special class of structured systems and derives explicit structured condition numbers for enhanced error estimation.

Findings

The proposed methods outperform standard conjugate gradient in robustness and accuracy.

Structured condition numbers lead to better forward error estimates.

Methods are competitive with direct solutions in accuracy.

Abstract

Given a full column rank matrix $A \in \mathbb{R}^{m\times n}$ ($m\geq n$), we consider a special class of linear systems of the form $A^\top Ax=A^\top b+c$ with $x, c \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^{m}$. The occurrence of $c$ in the right-hand side of the equation prevents the direct application of standard methods for least squares problems. Hence, we investigate alternative solution methods that, as in the case of normal equations, take advantage of the peculiar structure of the system to avoid unstable computations, such as forming $A^\top A$ explicitly. We propose two iterative methods that are based on specific reformulations of the problem and we provide explicit closed formulas for the structured condition number related to each problem. These formula allow us to compute a more accurate estimate of the forward error than the standard one used for generic linear systems, that does not take into account the structure of the perturbations. The relevance of our estimates is shown on a set of synthetic test problems. Numerical experiments highlight both the increased robustness and accuracy of the proposed methods compared to the standard conjugate gradient method. It is also found that the new methods can compare to standard direct methods in terms of solution accuracy.