Accelerating the Distributed Kaczmarz Algorithm by Strong Over-relaxation

TL;DR

This paper enhances the distributed Kaczmarz algorithm by allowing larger relaxation parameters, leading to faster convergence, especially in networked systems with tree structures, supported by theoretical analysis and numerical experiments.

Contribution

It introduces a method to use strong over-relaxation in the distributed Kaczmarz algorithm, expanding the range of relaxation parameters for improved convergence.

Findings

Larger relaxation parameters can be used than previously established.

The algorithm converges to the minimal norm solution in consistent systems.

Numerical experiments confirm the theoretical improvements.

Abstract

The distributed Kaczmarz algorithm is an adaptation of the standard Kaczmarz algorithm to the situation in which data is distributed throughout a network represented by a tree. We isolate substructures of the network and study convergence of the distributed Kazmarz algorithm for relatively large relaxation parameters associated to these substructures. If the system is consistent, then the algorithm converges to the solution of minimal norm; however, if the system is inconsistent, then the algorithm converges to an approximated least-squares solution that is dependent on the parameters and the network topology. We show that the relaxation parameters may be larger than the standard upper-bound in literature in this context and provide numerical experiments to support our results.