A fast randomized algorithm for orthogonal projection

TL;DR

This paper introduces a fast randomized algorithm for computing orthogonal projections onto the null and row spaces of a full-rank matrix, leveraging rapid application of the matrix and its adjoint, with high-probability preconditioner generation.

Contribution

The paper presents a novel randomized preconditioning approach that stabilizes and accelerates orthogonal projection computations for matrices with fewer rows than columns.

Findings

Rapid computation of projections with high probability success

Applicable to linear least-squares problems and convex optimization

Potential acceleration of interior-point methods

Abstract

We describe an algorithm that, given any full-rank matrix A having fewer rows than columns, can rapidly compute the orthogonal projection of any vector onto the null space of A, as well as the orthogonal projection onto the row space of A, provided that both A and its adjoint can be applied rapidly to arbitrary vectors. As an intermediate step, the algorithm solves the overdetermined linear least-squares regression involving the adjoint of A (and so can be used for this, too). The basis of the algorithm is an obvious but numerically unstable scheme; suitable use of a preconditioner yields numerical stability. We generate the preconditioner rapidly via a randomized procedure that succeeds with extremely high probability. In many circumstances, the method can accelerate interior-point methods for convex optimization, such as linear programming (Ming Gu, personal communication).