Quantile-Based Random Kaczmarz for corrupted linear systems of equations

TL;DR

This paper extends the quantile-based Random Kaczmarz method to deterministic settings, proving convergence for all perturbations below a certain threshold, and demonstrates its effectiveness on Gaussian matrices with minimal corruption.

Contribution

It provides a deterministic convergence guarantee for the quantile-based Random Kaczmarz method applicable to any matrix, improving robustness against corrupted data.

Findings

Convergence proven for all perturbations below a matrix-dependent threshold.

Method effective on tall Gaussian matrices with up to approximately 0.5% corruption.

Potential for improved thresholds with further analysis.

Abstract

We consider linear systems $Ax = b$ where $A \in \mathbb{R}^{m \times n}$ consists of normalized rows, $\|a_i\|_{\ell^2} = 1$, and where up to $\beta m$ entries of $b$ have been corrupted (possibly by arbitrarily large numbers). Haddock, Needell, Rebrova and Swartworth propose a quantile-based Random Kaczmarz method and show that for certain random matrices $A$ it converges with high likelihood to the true solution. We prove a deterministic version by constructing, for any matrix $A$, a number $\beta_A$ such that there is convergence for all perturbations with $\beta < \beta_A$. Assuming a random matrix heuristic, this proves convergence for tall Gaussian matrices with up to $\sim 0.5\%$ corruption (a number that can likely be improved).