Parallel inversion of huge covariance matrices

TL;DR

This paper introduces parallel algorithms for efficiently inverting large, nearly rank-deficient covariance matrices, significantly accelerating Gaussian process computations on multicore and cloud platforms.

Contribution

The paper presents a novel class of parallel algorithms for inverting large positive definite matrices, addressing a key bottleneck in statistical learning.

Findings

Achieved orders of magnitude speedup in matrix inversion

Successfully implemented on cloud computing platforms

Demonstrated practical benefits in Gaussian process applications

Abstract

An extremely common bottleneck encountered in statistical learning algorithms is inversion of huge covariance matrices, examples being in evaluating Gaussian likelihoods for a large number of data points. We propose general parallel algorithms for inverting positive definite matrices, which are nearly rank deficient. Such matrix inversions are needed in Gaussian process computations, among other settings, and remain a bottleneck even with the increasing literature on low rank approximations. We propose a general class of algorithms for parallelizing computations to dramatically speed up computation time by orders of magnitude exploiting multicore architectures. We implement our algorithm on a cloud computing platform, providing pseudo and actual code. The algorithm can be easily implemented on any multicore parallel computing resource. Some illustrations are provided to give a flavor for the gains and what becomes possible in freeing up this bottleneck.