Randomized Block Adaptive Linear System Solvers

TL;DR

This paper develops a theoretical framework for randomized adaptive linear system solvers, ensuring guaranteed convergence rates and enabling the design of efficient, verifiable algorithms for various types of linear systems.

Contribution

It introduces three general criteria for randomized adaptive solvers that guarantee exponential convergence, and demonstrates their application on twenty-six solvers, including nine novel methods.

Findings

Guaranteed exponential convergence for a broad class of solvers

Design principles for effective randomized adaptive solvers

Validation on twenty-six solvers, including nine new methods

Abstract

Randomized linear solvers randomly compress and solve a linear system with compelling theoretical convergence rates and computational complexities. However, such solvers suffer a substantial disconnect between their theoretical rates and actual efficiency in practice. Fortunately, these solvers are quite flexible and can be adapted to specific problems and computing environments to ensure high efficiency in practice, even at the cost of lower effectiveness (i.e., having a slower theoretical rate of convergence). While highly efficient adapted solvers can be readily designed by application experts, will such solvers still converge and at what rate? To answer this, we distill three general criteria for randomized adaptive solvers, which, as we show, will guarantee a worst-case exponential rate of convergence of the solver applied to consistent and inconsistent linear systems irrespective of whether such systems are over-determined, under-determined or rank-deficient. As a result, we enable application experts to design randomized adaptive solvers that achieve efficiency and can be verified for effectiveness using our theory. We demonstrate our theory on twenty-six solvers, nine of which are novel or novel block extensions of existing methods to the best of our knowledge.