Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression

TL;DR

This paper introduces a new aggregation model for large-scale Gaussian process regression that ensures consistency and efficiency, outperforming existing methods in accuracy and scalability.

Contribution

The paper proves the inconsistency of existing aggregation methods and proposes a novel, consistent, and efficient aggregation model for large-scale GP regression.

Findings

The new model provides consistent predictions that converge to the true function.

It maintains closed-form inference and supports parallel and distributed computing.

Empirical results show improved performance over state-of-the-art methods.

Abstract

In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregation process. We first prove the inconsistency of typical aggregations using disjoint or random data partition, and then present a consistent yet efficient aggregation model for large-scale GP. The proposed model inherits the advantages of aggregations, e.g., closed-form inference and aggregation, parallelization and distributed computing. Furthermore, theoretical and empirical analyses reveal that the new aggregation model performs better due to the consistent predictions that converge to the true underlying function when the training size approaches infinity.