Mixtures of Gaussian process experts based on kernel stick-breaking processes

TL;DR

This paper introduces a new mixture model of Gaussian process experts using kernel stick-breaking processes, enhancing predictive performance over existing models while maintaining interpretability.

Contribution

It proposes a novel Gaussian process mixture model based on kernel stick-breaking processes, improving predictive accuracy and scalability compared to Dirichlet process-based models.

Findings

Improved predictive performance demonstrated on six datasets.

Maintains interpretability and automatic expert selection.

Provides a practical slice sampling method for inference.

Abstract

Mixtures of Gaussian process experts is a class of models that can simultaneously address two of the key limitations inherent in standard Gaussian processes: scalability and predictive performance. In particular, models that use Dirichlet processes as gating functions permit straightforward interpretation and automatic selection of the number of experts in a mixture. While the existing models are intuitive and capable of capturing non-stationarity, multi-modality and heteroskedasticity, the simplicity of their gating functions may limit the predictive performance when applied to complex data-generating processes. Capitalising on the recent advancement in the dependent Dirichlet processes literature, we propose a new mixture model of Gaussian process experts based on kernel stick-breaking processes. Our model maintains the intuitive appeal yet improve the performance of the existing models. To make it practical, we design a sampler for posterior computation based on the slice sampling. The model behaviour and improved predictive performance are demonstrated in experiments using six datasets.