# Iterative methods for solving factorized linear systems

## TL;DR

This paper introduces a new iterative algorithm for solving large linear systems stored in factorized form, achieving faster convergence by leveraging the matrix factorization without explicitly computing the full matrix.

## Contribution

It proposes a novel randomized Kaczmarz variant that exploits matrix factorization, providing theoretical convergence guarantees and practical acceleration.

## Key findings

- Exponential convergence rate proven for the proposed method.
- Significant acceleration observed in experiments.
- Method effectively handles large low-rank datasets.

## Abstract

Stochastic iterative algorithms such as the Kaczmarz and Gauss-Seidel methods have gained recent attention because of their speed, simplicity, and the ability to approximately solve large-scale linear systems of equations without needing to access the entire matrix. In this work, we consider the setting where we wish to solve a linear system in a large matrix X that is stored in a factorized form, X = UV; this setting either arises naturally in many applications or may be imposed when working with large low-rank datasets for reasons of space required for storage. We propose a variant of the randomized Kaczmarz method for such systems that takes advantage of the factored form, and avoids computing X. We prove an exponential convergence rate and supplement our theoretical guarantees with experimental evidence demonstrating that the factored variant yields significant acceleration in convergence.