Sparse Sampling Kaczmarz-Motzkin Method with Linear Convergence

TL;DR

This paper introduces a greedy variant of the randomized sparse Kaczmarz method using the sampling Kaczmarz-Motzkin approach, proving its linear convergence and demonstrating improved performance through experiments.

Contribution

It proposes a new greedy sampling Kaczmarz-Motzkin method that unifies existing approaches and achieves linear convergence in sparse linear system recovery.

Findings

Proves linear convergence in expectation for the new method

Demonstrates superior numerical performance over existing methods

Unifies sampling Kaczmarz-Motzkin and randomized sparse Kaczmarz approaches

Abstract

The randomized sparse Kaczmarz method was recently proposed to recover sparse solutions of linear systems. In this work, we introduce a greedy variant of the randomized sparse Kaczmarz method by employing the sampling Kaczmarz-Motzkin method, and prove its linear convergence in expectation with respect to the Bregman distance in the noiseless and noisy cases. This greedy variant can be viewed as a unification of the sampling Kaczmarz-Motzkin method and the randomized sparse Kaczmarz method, and hence inherits the merits of these two methods. Numerically, we report a couple of experimental results to demonstrate its superiority