Scalable Bayesian Transformed Gaussian Processes

TL;DR

This paper introduces scalable methods for Bayesian Transformed Gaussian Processes, enabling practical high-dimensional regression with improved accuracy and uncertainty quantification compared to traditional maximum likelihood approaches.

Contribution

It develops fast, principled techniques for Bayesian BTG, making it feasible for high-dimensional data and layered transformations, enhancing expressibility and performance.

Findings

BTG achieves superior empirical performance over MLE-based models.

The proposed methods enable fast prediction and model selection in high dimensions.

Layered transformations significantly improve BTG expressibility.

Abstract

The Bayesian transformed Gaussian process (BTG) model, proposed by Kedem and Oliviera, is a fully Bayesian counterpart to the warped Gaussian process (WGP) and marginalizes out a joint prior over input warping and kernel hyperparameters. This fully Bayesian treatment of hyperparameters often provides more accurate regression estimates and superior uncertainty propagation, but is prohibitively expensive. The BTG posterior predictive distribution, itself estimated through high-dimensional integration, must be inverted in order to perform model prediction. To make the Bayesian approach practical and comparable in speed to maximum-likelihood estimation (MLE), we propose principled and fast techniques for computing with BTG. Our framework uses doubly sparse quadrature rules, tight quantile bounds, and rank-one matrix algebra to enable both fast model prediction and model selection. These scalable methods allow us to regress over higher-dimensional datasets and apply BTG with layered transformations that greatly improve its expressibility. We demonstrate that BTG achieves superior empirical performance over MLE-based models.