A federated Kaczmarz algorithm

TL;DR

This paper introduces a federated Kaczmarz algorithm for large linear systems, providing convergence guarantees, experimental validation, and applications to sparse approximation, inconsistent systems, and real data, highlighting its efficiency and versatility.

Contribution

The paper presents a novel federated Kaczmarz algorithm with proven convergence and demonstrates its effectiveness in sparse, inconsistent, and real-world data scenarios.

Findings

Convergence guarantees for the federated Kaczmarz algorithm.

Effective sparse approximation in underdetermined systems.

Consistent convergence to least squares solutions in inconsistent systems.

Abstract

In this paper, we propose a federated algorithm for solving large linear systems that is inspired by the classic randomized Kaczmarz algorithm. We provide convergence guarantees of the proposed method, and as a corollary of our analysis, we provide a new proof for the convergence of the classic randomized Kaczmarz method. We demonstrate experimentally the behavior of our method when applied to related problems. For underdetermined systems, we demonstrate that our algorithm can be used for sparse approximation. For inconsistent systems, we demonstrate that our algorithm converges to a horizon of the least squares solution. Finally, we apply our algorithm to real data and show that it is consistent with the selection of Lasso, while still offering the computational advantages of the Kaczmarz framework and thresholding-based algorithms in the federated setting.