Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise

TL;DR

This paper extends existing analysis of QR and related algorithms to account for Gaussian noise and imperfect orthogonality, providing bounds on condition number changes and performance in noisy environments.

Contribution

It introduces bounds on the condition number increase for matrices with non-orthogonal columns under Gaussian noise, relaxing previous orthonormality assumptions.

Findings

Derived bounds on condition number increase with noisy orthogonalization

Analyzed performance of orthogonal projection under Gaussian noise

Extended theoretical understanding of least squares in noisy settings

Abstract

In this paper, we extend the work of Liesen et al. (2002), which analyzes how the condition number of an orthonormal matrix Q changes when a column is added ([Q, c]), particularly focusing on the perpendicularity of c to the span of Q. Their result, presented in Theorem 2.3 of Liesen et al. (2002), assumes exact arithmetic and orthonormality of Q, which is a strong assumption when applying these results to numerical methods such as QR factorization algorithms. In our work, we address this gap by deriving bounds on the condition number increase for a matrix B without assuming perfect orthonormality, even when a column is not perfectly orthogonal to the span of B. This framework allows us to analyze QR factorization methods where orthogonalization is imperfect and subject to Gaussian noise. We also provide results on the performance of orthogonal projection and least squares under Gaussian noise, further supporting the development of this theory.