Penalty & Augmented Kaczmarz Methods For Linear Systems & Linear Feasibility Problems

TL;DR

This paper introduces two novel variants of the Randomized Kaczmarz method, incorporating penalty and augmented Lagrangian techniques, with proven linear convergence for solving linear systems and feasibility problems.

Contribution

It proposes RPK and RAK methods that integrate optimization techniques into Kaczmarz, providing new algorithms with convergence guarantees.

Findings

Both RPK and RAK achieve linear convergence.

The methods effectively solve linear systems and feasibility problems.

Theoretical analysis confirms convergence properties.

Abstract

In this work, we shed light on the so-called Kaczmarz method for solving Linear System (LS) and Linear Feasibility (LF) problems from a optimization point of view. We introduce well-known optimization approaches such as Lagrangian penalty and Augmented Lagrangian in the Randomized Kaczmarz (RK) method. In doing so, we propose two variants of the RK method namely the Randomized Penalty Kacmarz (RPK) method and Randomized Augmented Kacmarz (RAK) method. We carry out convergence analysis of the proposed methods and obtain linear convergence results.