Online Signal Recovery via Heavy Ball Kaczmarz

TL;DR

This paper introduces a heavy ball momentum variant of the Kaczmarz method for online signal recovery, demonstrating accelerated convergence especially with coherent data, supported by theoretical guarantees.

Contribution

It proposes a novel heavy ball momentum approach for the Kaczmarz method in online settings, with theoretical analysis showing improved convergence rates.

Findings

Heavy ball momentum accelerates convergence with coherent data.

Theoretical linear convergence guarantee for a broad class of source distributions.

Enhanced performance over standard Kaczmarz in online signal recovery.

Abstract

Recovering a signal $x^\ast \in \mathbb{R}^n$ from a sequence of linear measurements is an important problem in areas such as computerized tomography and compressed sensing. In this work, we consider an online setting in which measurements are sampled one-by-one from some source distribution. We propose solving this problem with a variant of the Kaczmarz method with an additional heavy ball momentum term. A popular technique for solving systems of linear equations, recent work has shown that the Kaczmarz method also enjoys linear convergence when applied to random measurement models, however convergence may be slowed when successive measurements are highly coherent. We demonstrate that the addition of heavy ball momentum may accelerate the convergence of the Kaczmarz method when data is coherent, and provide a theoretical analysis of the method culminating in a linear convergence guarantee for a wide class of source distributions.