Distributed Gaussian Processes

TL;DR

This paper introduces the robust Bayesian Committee Machine (rBCM), a scalable, simple, and distributed Gaussian process regression method that efficiently handles large datasets without relying on sparse approximations.

Contribution

The paper presents the rBCM, a novel distributed GP regression model that is conceptually simple, does not require inducing points, and is suitable for heterogeneous and large-scale computing environments.

Findings

rBCM enables scalable GP regression on large datasets.

The method allows efficient parallel and distributed computation.

It performs well across different computing infrastructures.

Abstract

To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of-the-art sparse GP approximations, the rBCM is conceptually simple and does not rely on inducing or variational parameters. The key idea is to recursively distribute computations to independent computational units and, subsequently, recombine them to form an overall result. Efficient closed-form inference allows for straightforward parallelisation and distributed computations with a small memory footprint. The rBCM is independent of the computational graph and can be used on heterogeneous computing infrastructures, ranging from laptops to clusters. With sufficient computing resources our distributed GP model can handle arbitrarily large data sets.