Sparse matrix factorizations for fast linear solvers with application to Laplacian systems

TL;DR

This paper introduces a new sparse matrix factorization technique that enables fast iterative solutions for Laplacian systems, bridging the gap between cheap sparse and expensive dense search directions, with applications to graph-based linear solvers.

Contribution

It proposes a novel sparse factorization method allowing efficient non-sparse search directions, leading to nearly-linear time algorithms for Laplacian system solutions.

Findings

Nearly-linear time algorithm for minimal norm solutions of graph Laplacians.

Connection to existing nearly-linear Laplacian solvers through sparse factorizations.

Efficient iterative methods using hierarchical matrix search directions.

Abstract

In solving a linear system with iterative methods, one is usually confronted with the dilemma of having to choose between cheap, inefficient iterates over sparse search directions (e.g., coordinate descent), or expensive iterates in well-chosen search directions (e.g., conjugate gradients). In this paper, we propose to interpolate between these two extremes, and show how to perform cheap iterations along non-sparse search directions, provided that these directions can be extracted from a new kind of sparse factorization. For example, if the search directions are the columns of a hierarchical matrix, then the cost of each iteration is typically logarithmic in the number of variables. Using some graph-theoretical results on low-stretch spanning trees, we deduce as a special case a nearly-linear time algorithm to approximate the minimal norm solution of a linear system $Bx= b$ where $B$ is the incidence matrix of a graph. We thereby can connect our results to recently proposed nearly-linear time solvers for Laplacian systems, which emerge here as a particular application of our sparse matrix factorization.