Adaptive Randomized Extended Bregman-Kaczmarz Method for Combined Optimization Problems

TL;DR

This paper introduces an adaptive randomized extended Bregman-Kaczmarz method for solving combined optimization problems, demonstrating faster convergence and robustness through theoretical guarantees and numerical experiments.

Contribution

It proposes an adaptive, automatically tuned algorithm with proven convergence for combined optimization problems, improving over existing methods.

Findings

Achieves faster convergence than state-of-the-art algorithms.

Demonstrates robustness on synthetic and real data.

Provides theoretical linear convergence guarantees.

Abstract

Combined optimization problems that couple data-fidelity and regularization terms arise naturally in a wide range of inverse problems. In this paper, we study an adaptive randomized averaging block extended Bregman-Kaczmarz (aRABEBK) method for solving such problems. The proposed method incorporates iteration-wise relaxation parameters that are automatically adjusted using residual information, allowing for more aggressive step sizes without additional manual tuning. We establish a convergence theory for the proposed framework and derive expected linear convergence rate guarantees. Numerical experiments on both synthetic and real data sets for sparse and minimum-norm least-squares problems demonstrate that our aRABEBK method achieves faster convergence and improved robustness compared with state-of-the-art extended Kaczmarz and Bregman-Kaczmarz-type algorithms, including its nonadaptive counterpart.