Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation

TL;DR

This paper introduces a scalable stochastic variational method for Gaussian Process classification using Polya-Gamma augmentation, achieving significant speedups while maintaining competitive accuracy on large datasets.

Contribution

The paper presents a novel closed-form update approach with natural gradients for GP classification, enabling efficient training on very large datasets.

Findings

Up to 100x faster than existing methods on large datasets

Maintains competitive prediction accuracy

Successfully scales to datasets with over 11 million points

Abstract

We propose a scalable stochastic variational approach to GP classification building on Polya-Gamma data augmentation and inducing points. Unlike former approaches, we obtain closed-form updates based on natural gradients that lead to efficient optimization. We evaluate the algorithm on real-world datasets containing up to 11 million data points and demonstrate that it is up to two orders of magnitude faster than the state-of-the-art while being competitive in terms of prediction performance.