Gaussian Graphical Models as an Ensemble Method for Distributed Gaussian Processes

TL;DR

This paper introduces a novel ensemble method for distributed Gaussian processes using Gaussian graphical models, improving aggregation of local predictions and outperforming existing approaches on synthetic and real data.

Contribution

It proposes a new aggregation technique for distributed Gaussian processes based on Gaussian graphical models, addressing limitations of the independence assumption.

Findings

Outperforms state-of-the-art DGP methods on various datasets

Effectively models interactions between local experts

Uses EM algorithm for joint distribution estimation

Abstract

Distributed Gaussian process (DGP) is a popular approach to scale GP to big data which divides the training data into some subsets, performs local inference for each partition, and aggregates the results to acquire global prediction. To combine the local predictions, the conditional independence assumption is used which basically means there is a perfect diversity between the subsets. Although it keeps the aggregation tractable, it is often violated in practice and generally yields poor results. In this paper, we propose a novel approach for aggregating the Gaussian experts' predictions by Gaussian graphical model (GGM) where the target aggregation is defined as an unobserved latent variable and the local predictions are the observed variables. We first estimate the joint distribution of latent and observed variables using the Expectation-Maximization (EM) algorithm. The interaction between experts can be encoded by the precision matrix of the joint distribution and the aggregated predictions are obtained based on the property of conditional Gaussian distribution. Using both synthetic and real datasets, our experimental evaluations illustrate that our new method outperforms other state-of-the-art DGP approaches.