Adaptive Multiple Importance Sampling for Gaussian Processes

TL;DR

This paper introduces an Adaptive Multiple Importance Sampling framework for Gaussian processes, offering an efficient alternative to MCMC for posterior inference of covariance parameters, especially with non-Gaussian likelihoods.

Contribution

It develops a novel AMIS-based inference method for Gaussian process covariance parameters, including a Pseudo-Marginal approach for non-Gaussian likelihoods, improving efficiency over traditional MCMC.

Findings

Outperforms MCMC in various scenarios

Effective for non-Gaussian likelihoods

Remains competitive in moderate dimensions

Abstract

In applications of Gaussian processes where quantification of uncertainty is a strict requirement, it is necessary to accurately characterize the posterior distribution over Gaussian process covariance parameters. Normally, this is done by means of standard Markov chain Monte Carlo (MCMC) algorithms. Motivated by the issues related to the complexity of calculating the marginal likelihood that can make MCMC algorithms inefficient, this paper develops an alternative inference framework based on Adaptive Multiple Importance Sampling (AMIS). This paper studies the application of AMIS in the case of a Gaussian likelihood, and proposes the Pseudo-Marginal AMIS for non-Gaussian likelihoods, where the marginal likelihood is unbiasedly estimated. The results suggest that the proposed framework outperforms MCMC-based inference of covariance parameters in a wide range of scenarios and remains competitive for moderately large dimensional parameter spaces.