Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method

TL;DR

This paper introduces a randomized block Kaczmarz method for solving overdetermined least-squares problems, achieving a linear convergence rate based on matrix properties, and emphasizes the importance of good row pavings for efficiency.

Contribution

It presents the first block Kaczmarz algorithm with proven linear convergence rate tied to matrix geometry and highlights the role of well-conditioned row partitions.

Findings

The algorithm converges linearly with an expected rate.

Good row pavings significantly improve efficiency.

The method applies broadly to overdetermined least-squares problems.

Abstract

The block Kaczmarz method is an iterative scheme for solving overdetermined least-squares problems. At each step, the algorithm projects the current iterate onto the solution space of a subset of the constraints. This paper describes a block Kaczmarz algorithm that uses a randomized control scheme to choose the subset at each step. This algorithm is the first block Kaczmarz method with an (expected) linear rate of convergence that can be expressed in terms of the geometric properties of the matrix and its submatrices. The analysis reveals that the algorithm is most effective when it is given a good row paving of the matrix, a partition of the rows into well-conditioned blocks. The operator theory literature provides detailed information about the existence and construction of good row pavings. Together, these results yield an efficient block Kaczmarz scheme that applies to many overdetermined least-squares problem.