Surrounding the solution of a Linear System of Equations from all sides

TL;DR

The paper introduces a geometric method for solving linear systems by reflecting through hyperplanes, demonstrating convergence properties and bounds that are independent of matrix dimension, and compares this to existing averaging techniques.

Contribution

It presents a novel geometric approach using hyperplane reflections for solving linear systems, with convergence analysis and bounds similar to the Random Kaczmarz method.

Findings

Sequences generated have constant distance to the solution

Averaging over random hyperplane reflections converges at a rate of 1/√m

Bound on error does not depend on matrix dimension

Abstract

Suppose $A \in \mathbb{R}^{n \times n}$ is invertible and we are looking for the solution of $Ax = b$. Given an initial guess $x_1 \in \mathbb{R}$, we show that by reflecting through hyperplanes generated by the rows of $A$, we can generate an infinite sequence $(x_k)_{k=1}^{\infty}$ such that all elements have the same distance to the solution, i.e. $\|x_k - x\| = \|x_1 - x\|$. If the hyperplanes are chosen at random, averages over the sequence converge and $$ \mathbb{E} \left\| x - \frac{1}{m} \sum_{k=1}^{m}{ x_k} \right\| \leq \frac{1 + \|A\|_F \|A^{-1}\|}{\sqrt{m}} \cdot\|x-x_1\|.$$ The bound does not depend on the dimension of the matrix. This introduces a purely geometric way of attacking the problem: are there fast ways of estimating the location of the center of a sphere from knowing many points on the sphere? Our convergence rate (coinciding with that of the Random Kaczmarz method) comes from averaging, can one do better?