Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models

TL;DR

This paper introduces an automated augmented conjugate inference technique for non-conjugate Gaussian process models, enabling faster and more robust inference through auxiliary variable augmentation and two novel inference methods.

Contribution

It presents a new automated augmentation approach that transforms non-conjugate GP models into conditionally conjugate forms, facilitating efficient inference methods.

Findings

Significantly faster inference compared to existing methods.

More robust performance in various datasets.

Effective for both large-scale and small datasets.

Abstract

We propose automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models. Our method automatically constructs an auxiliary variable augmentation that renders the GP model conditionally conjugate. Building on the conjugate structure of the augmented model, we develop two inference methods. First, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form. Second, an asymptotically correct Gibbs sampler that is useful for small datasets. Our experiments show that our method are up two orders of magnitude faster and more robust than existing state-of-the-art black-box methods.