New Studies of Randomized Augmentation and Additive Preprocessing

TL;DR

This paper investigates the properties of randomized augmentation and additive preprocessing for matrices, demonstrating their effectiveness in improving rank, conditioning, and computational efficiency in matrix algorithms.

Contribution

It introduces new theoretical results on Gaussian random matrices, extends these to additive preprocessing, and develops faster, more efficient algorithms for matrix approximation using structured randomization.

Findings

Gaussian matrices are full rank and well-conditioned with high probability.

Augmentation with Gaussian rows or columns improves rank and conditioning even for ill-conditioned matrices.

The proposed algorithms are faster and require fewer random parameters, maintaining accuracy.

Abstract

1. A standard Gaussian random matrix has full rank with probability 1 and is well-conditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular. 2. If we append sufficiently many standard Gaussian random rows or columns to any normalized matrix A, then the augmented matrix has full rank with probability 1 and is well-conditioned with a probability close to 1, even if the matrix A is rank deficient or ill-conditioned. 3. We specify and prove these properties of augmentation and extend them to additive preprocessing, that is, to adding a product of two rectangular Gaussian matrices. 4. By applying our randomization techniques to a matrix that has numerical rank r, we accelerate the known algorithms for the approximation of its leading and trailing singular spaces associated with its r largest and with all its remaining singular values, respectively. 5. Our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace Gaussian. Empirically the outputs of the resulting algorithms is as accurate as the outputs under Gaussian preprocessing. 6. Our novel duality techniques provides formal support, so far missing, for these empirical observations and opens door to derandomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors. 7. Our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low-rank approximation of a matrix by means of random sampling.