Aggregating Dependent Gaussian Experts in Local Approximation

TL;DR

This paper introduces a novel method for aggregating Gaussian experts in distributed Gaussian processes by detecting dependencies using Gaussian graphical models, leading to improved accuracy and efficiency.

Contribution

It proposes a new aggregation technique that accounts for dependencies among experts, addressing limitations of the independence assumption in existing methods.

Findings

Outperforms state-of-the-art DGP methods in accuracy.

More time-efficient than existing independent expert approaches.

Effective in both synthetic and real datasets.

Abstract

Distributed Gaussian processes (DGPs) are prominent local approximation methods to scale Gaussian processes (GPs) to large datasets. Instead of a global estimation, they train local experts by dividing the training set into subsets, thus reducing the time complexity. This strategy is based on the conditional independence assumption, which basically means that there is a perfect diversity between the local experts. In practice, however, this assumption is often violated, and the aggregation of experts leads to sub-optimal and inconsistent solutions. In this paper, we propose a novel approach for aggregating the Gaussian experts by detecting strong violations of conditional independence. The dependency between experts is determined by using a Gaussian graphical model, which yields the precision matrix. The precision matrix encodes conditional dependencies between experts and is used to detect strongly dependent experts and construct an improved aggregation. Using both synthetic and real datasets, our experimental evaluations illustrate that our new method outperforms other state-of-the-art (SOTA) DGP approaches while being substantially more time-efficient than SOTA approaches, which build on independent experts.