Minimal error momentum Bregman-Kaczmarz

TL;DR

This paper introduces an adaptive heavy ball momentum technique for the Bregman-Kaczmarz method, improving convergence speed for large-scale convex problems with linear constraints, with theoretical guarantees and practical benefits demonstrated.

Contribution

It proposes an adaptive momentum parameter selection based on a minimal-error principle, enabling accelerated convergence analysis for the Bregman-Kaczmarz method.

Findings

The adaptive momentum method improves convergence over the non-accelerated version.

The method achieves theoretical optimality in error minimization.

Numerical experiments confirm practical acceleration, including in tomography applications.

Abstract

The Bregman-Kaczmarz method is an iterative method which can solve strongly convex problems with linear constraints and uses only one or a selected number of rows of the system matrix in each iteration, thereby making it amenable for large-scale systems. To speed up convergence, we investigate acceleration by heavy ball momentum in the so-called dual update. Heavy ball acceleration of the Kaczmarz method with constant parameters has turned out to be difficult to analyze, in particular no accelerated convergence for the L2-error of the iterates has been proven to the best of our knowledge. Here we propose a way to adaptively choose the momentum parameter by a minimal-error principle similar to a recently proposed method for the standard randomized Kaczmarz method. The momentum parameter can be chosen to exactly minimize the error in the next iterate or to minimize a relaxed version of the minimal error principle. The former choice leads to a theoretically optimal step while the latter is cheaper to compute. We prove improved convergence results compared to the non-accelerated method. Numerical experiments show that the proposed methods can accelerate convergence in practice, also for matrices which arise from applications such as computational tomography.