Correlated Product of Experts for Sparse Gaussian Process Regression

TL;DR

This paper introduces a scalable Gaussian process regression method that combines local experts with adjustable correlation, improving accuracy and efficiency over existing sparse GP techniques.

Contribution

It proposes a novel correlated product of experts framework that generalizes existing sparse GP methods and offers linear complexity in data size and number of experts.

Findings

Outperforms state-of-the-art GP approximations in accuracy and speed.

Handles general kernels and multiple variables effectively.

Maintains consistent uncertainty estimates across varying expert correlations.

Abstract

Gaussian processes (GPs) are an important tool in machine learning and statistics with applications ranging from social and natural science through engineering. They constitute a powerful kernelized non-parametric method with well-calibrated uncertainty estimates, however, off-the-shelf GP inference procedures are limited to datasets with several thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques have been developed over the past years. In this paper, we focus on GP regression tasks and propose a new approach based on aggregating predictions from several local and correlated experts. Thereby, the degree of correlation between the experts can vary between independent up to fully correlated experts. The individual predictions of the experts are aggregated taking into account their correlation resulting in consistent uncertainty estimates. Our method recovers independent Product of Experts, sparse GP and full GP in the limiting cases. The presented framework can deal with a general kernel function and multiple variables, and has a time and space complexity which is linear in the number of experts and data samples, which makes our approach highly scalable. We demonstrate superior performance, in a time vs. accuracy sense, of our proposed method against state-of-the-art GP approximation methods for synthetic as well as several real-world datasets with deterministic and stochastic optimization.