Bayesian Bridge Gaussian Process Regression

TL;DR

This paper introduces Bayesian Bridge Gaussian Process Regression (B2GPR), a novel method that applies sparsity-inducing priors to improve variable selection and prediction in high-dimensional Gaussian Process models.

Contribution

It develops a new Bayesian framework with $ ext{l}_q$-norm constraints for variable selection in GP regression, including a Gibbs sampler with Spherical HMC for constrained posteriors.

Findings

B2GPR outperforms existing methods in variable selection.

B2GPR achieves better predictive accuracy in simulations.

The method effectively identifies active variables in real data.

Abstract

The performance of Gaussian Process (GP) regression is often hampered by the curse of dimensionality, which inflates computational cost and reduces predictive power in high-dimensional problems. Variable selection is thus crucial for building efficient and accurate GP models. Inspired by Bayesian bridge regression, we propose the Bayesian Bridge Gaussian Process Regression (B\textsuperscript{2}GPR) model. This framework places $\ell_q$-norm constraints on key GP parameters to automatically induce sparsity and identify active variables. We formulate two distinct versions: one for $q=2$ using conjugate Gaussian priors, and another for $0<q<2$ that employs constrained flat priors, leading to non-standard, norm-constrained posterior distributions. To enable posterior inference, we design a Gibbs sampling algorithm that integrates Spherical Hamiltonian Monte Carlo (SphHMC) to efficiently sample from the constrained posteriors when $0<q<2$. Simulations and a real-data application confirm that B\textsuperscript{2}GPR offers superior variable selection and prediction compared to alternative approaches.