Sketching for Motzkin's Iterative Method for Linear Systems

TL;DR

This paper demonstrates that Gaussian sketching enhances the convergence and theoretical guarantees of Motzkin's iterative method for solving large linear systems, supported by numerical experiments.

Contribution

It introduces a novel analysis showing accelerated convergence of the sketched relaxation method using Gaussian sketching, applicable to arbitrary linear systems.

Findings

Gaussian sketching improves convergence rates

Theoretical guarantees are strengthened for arbitrary systems

Numerical experiments support the theoretical results

Abstract

Projection-based iterative methods for solving large over-determined linear systems are well-known for their simplicity and computational efficiency. It is also known that the correct choice of a sketching procedure (i.e., preprocessing steps that reduce the dimension of each iteration) can improve the performance of iterative methods in multiple ways, such as, to speed up the convergence of the method by fighting inner correlations of the system, or to reduce the variance incurred by the presence of noise. In the current work, we show that sketching can also help us to get better theoretical guarantees for the projection-based methods. Specifically, we use good properties of Gaussian sketching to prove an accelerated convergence rate of the sketched relaxation (also known as Motzkin's) method. The new estimates hold for linear systems of arbitrary structure. We also provide numerical experiments in support of our theoretical analysis of the sketched relaxation method.