Well-Conditioned Oblivious Perturbations in Linear Space

TL;DR

This paper introduces a computationally efficient method for perturbing matrices to improve their condition number, enabling faster and more space-efficient solutions to linear systems while maintaining accuracy.

Contribution

It proposes a novel perturbation technique requiring only O(n) random numbers, matching Gaussian perturbations in effectiveness but with lower computational cost.

Findings

Reduces the condition number of matrices to O(n) using minimal randomness.

Enables solving linear systems in linear space with O(n) matrix-vector products.

Develops new techniques for analyzing singular values of dependent random matrices.

Abstract

Perturbing a deterministic $n$-dimensional matrix with small Gaussian noise is a cornerstone of smoothed analysis of algorithms [Spielman and Teng, JACM 2004], as it reduces the condition number of the input to $O(n)$, and with it the complexity of many matrix algorithms. However, when deployed algorithmically, these perturbations are expensive due to the cost of generating and storing $n^2$ Gaussian random variables. We propose a perturbation that requires generating and storing $O(n)$ random numbers in $O(\log n)$ bits of precision, and reduces the condition number of any deterministic matrix to $O(n)$, matching Gaussian perturbations. Our result in particular implies a better complexity for the perturbed conjugate gradient algorithm, showing that we can solve an $n\times n$ linear system in linear space to within an arbitrarily small constant backward error using $O(n)$ matrix-vector products. In our construction, we introduce the concept of a pattern matrix, which is a dense deterministic matrix that maps all sparse vectors into dense vectors, and we combine it with a sparse perturbation whose entries are dependent and located in a non-uniform fashion. In order to analyze this construction, we develop new techniques for lower bounding the smallest singular value of a random matrix with dependent entries.