Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

TL;DR

This paper introduces an extended sparse randomized Kaczmarz method that efficiently finds sparse least squares solutions in systems with impulsive noise, combining ideas from existing methods and proving convergence.

Contribution

It proposes a novel extended sparse randomized Kaczmarz method that converges linearly to sparse solutions and handles impulsive noise, extending previous approaches.

Findings

Method converges linearly to sparse least squares solutions.

Effective in systems with impulsive noise.

Numerical experiments confirm robustness and efficiency.

Abstract

The Extended Randomized Kaczmarz method is a well known iterative scheme which can find the Moore-Penrose inverse solution of a possibly inconsistent linear system and requires only one additional column of the system matrix in each iteration in comparison with the standard randomized Kaczmarz method. Also, the Sparse Randomized Kaczmarz method has been shown to converge linearly to a sparse solution of a consistent linear system. Here, we combine both ideas and propose an Extended Sparse Randomized Kaczmarz method. We show linear expected convergence to a sparse least squares solution in the sense that an extended variant of the regularized basis pursuit problem is solved. Moreover, we generalize the additional step in the method and prove convergence to a more abstract optimization problem. We demonstrate numerically that our method can find sparse least squares solutions of real and complex systems if the noise is concentrated in the complement of the range of the system matrix and that our generalization can handle impulsive noise.