Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo

TL;DR

This paper introduces a fully non-stationary Gaussian process regression method that models input-dependent noise, signal variance, and lengthscale, using Hamiltonian Monte Carlo for inference, enabling more realistic modeling of complex data.

Contribution

It develops a novel non-stationary GPR framework with gradient-based inference and HMC, allowing for flexible modeling of input-dependent parameters without approximations.

Findings

Non-stationary GPR captures realistic input-dependent dynamics.

HMC enables full posterior inference for non-stationary parameters.

Performs comparably to traditional models on synthetic and gene expression data.

Abstract

We present a novel approach for fully non-stationary Gaussian process regression (GPR), where all three key parameters -- noise variance, signal variance and lengthscale -- can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. We propose to infer full parameter posterior with Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with model gradients. We also learn the MAP solution from the posterior by gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the nonstationary GPR is shown to be necessary for modeling realistic input-dependent dynamics, while it performs comparably to conventional stationary or previous non-stationary GPR models otherwise.