Connecting randomized iterative methods with Krylov subspaces

TL;DR

This paper establishes a theoretical and practical connection between randomized iterative methods and Krylov subspace algorithms, leading to new efficient hybrid methods with proven linear convergence.

Contribution

It introduces a unified framework linking randomized methods with Krylov techniques, enabling the design of novel iterative-sketching Krylov algorithms with theoretical guarantees.

Findings

Proves linear convergence of the proposed methods in expectation

Develops a new class of iterative-sketching Krylov algorithms

Validates the approach with numerical experiments

Abstract

Randomized iterative methods, such as the randomized Kaczmarz method, have gained significant attention for solving large-scale linear systems due to their simplicity and efficiency. Meanwhile, Krylov subspace methods have emerged as a powerful class of algorithms, known for their robust theoretical foundations and rapid convergence properties. Despite the individual successes of these two paradigms, their underlying connection has remained largely unexplored. In this paper, we develop a unified framework that bridges randomized iterative methods and Krylov subspace techniques, supported by both rigorous theoretical analysis and practical implementation. The core idea is to formulate each iteration as an adaptively weighted linear combination of the sketched normal vector and previous iterates, with the weights optimally determined via a projection-based mechanism. This formulation not only reveals how subspace techniques can enhance the efficiency of randomized iterative methods, but also enables the design of a new class of iterative-sketching-based Krylov subspace algorithms. We prove that our method converges linearly in expectation and validate our findings with numerical experiments.