A fast semi-direct least squares algorithm for hierarchically block separable matrices

TL;DR

This paper introduces a fast, semi-direct algorithm for solving least squares problems involving hierarchically block separable matrices, leveraging sparsity and recursive skeletonization to achieve optimal complexity in various dimensions.

Contribution

The authors develop a novel recursive skeletonization-based method that efficiently solves dense least squares problems with HBS matrices by combining matrix compression, sparse QR factorization, and iterative constraints.

Findings

Achieves optimal $\\mathcal{O}(M+N)$ complexity in 1D.

Extends to higher dimensions with increased complexity, e.g., $\mathcal{O}(M+N^{2})$ in 3D.

Effective for radial basis function approximation and matrix updates.

Abstract

We present a fast algorithm for linear least squares problems governed by hierarchically block separable (HBS) matrices. Such matrices are generally dense but data-sparse and can describe many important operators including those derived from asymptotically smooth radial kernels that are not too oscillatory. The algorithm is based on a recursive skeletonization procedure that exposes this sparsity and solves the dense least squares problem as a larger, equality-constrained, sparse one. It relies on a sparse QR factorization coupled with iterative weighted least squares methods. In essence, our scheme consists of a direct component, comprised of matrix compression and factorization, followed by an iterative component to enforce certain equality constraints. At most two iterations are typically required for problems that are not too ill-conditioned. For an $M \times N$ HBS matrix with $M \geq N$ having bounded off-diagonal block rank, the algorithm has optimal $\mathcal{O} (M + N)$ complexity. If the rank increases with the spatial dimension as is common for operators that are singular at the origin, then this becomes $\mathcal{O} (M + N)$ in 1D, $\mathcal{O} (M + N^{3/2})$ in 2D, and $\mathcal{O} (M + N^{2})$ in 3D. We illustrate the performance of the method on both over- and underdetermined systems in a variety of settings, with an emphasis on radial basis function approximation and efficient updating and downdating.