On Subsampled Quantile Randomized Kaczmarz

TL;DR

This paper introduces and analyzes the sub-sampled quantile Randomized Kaczmarz (sQRK) algorithm, which efficiently solves large noisy linear systems by approximating quantile thresholds with residual sub-sampling, ensuring convergence despite noise.

Contribution

It provides a theoretical analysis of sQRK, demonstrating convergence and exploring the effects of sample size and quantile choice in large-scale noisy linear systems.

Findings

sQRK converges to the true solution under certain conditions

The method is effective with small sample sizes

Numerical experiments support theoretical results

Abstract

When solving noisy linear systems Ax = b + c, the theoretical and empirical performance of stochastic iterative methods, such as the Randomized Kaczmarz algorithm, depends on the noise level. However, if there are a small number of highly corrupt measurements, one can instead use quantile-based methods to guarantee convergence to the solution x of the system, despite the presence of noise. Such methods require the computation of the entire residual vector, which may not be desirable or even feasible in some cases. In this work, we analyze the sub-sampled quantile Randomized Kaczmarz (sQRK) algorithm for solving large-scale linear systems which utilize a sub-sampled residual to approximate the quantile threshold. We prove that this method converges to the unique solution to the linear system and provide numerical experiments that support our theoretical findings. We additionally remark on the extremely small sample size case and demonstrate the importance of interplay between the choice of quantile and subset size.