IID Sampling from Doubly Intractable Distributions

TL;DR

This paper introduces an efficient iid sampling method for doubly intractable posteriors using Monte Carlo, importance sampling, and Gaussian process interpolation, demonstrating high accuracy and computational efficiency in complex models.

Contribution

It develops a novel iid sampling approach for doubly intractable distributions by combining Monte Carlo approximations, Gaussian process interpolation, and the framework from Bhattacharya (2021a, 2021b).

Findings

High accuracy in diverse models including high-dimensional cases.

Computationally efficient, generating 10,000 iid samples in minutes.

Effective in models like Ising, Strauss, and autologistic.

Abstract

Intractable posterior distributions of parameters with intractable normalizing constants depending upon the parameters are known as doubly intractable posterior distributions. The terminology itself indicates that obtaining Bayesian inference from such posteriors is doubly difficult compared to traditional intractable posteriors where the normalizing constants are tractable and admit traditional Markov Chain Monte Carlo (MCMC) solutions. As can be anticipated, a plethora of MCMC-based methods have originated in the literature to deal with doubly intractable distributions. Yet, it remains very much unclear if any of the methods can satisfactorily sample from such posteriors, particularly in high-dimensional setups. In this article, we consider efficient Monte Carlo and importance sampling approximations of the intractable normalizing constant for a few values of the parameters, and Gaussian process interpolations for the remaining values of the parameters, using the approximations. We then incorporate this strategy within the exact iid sampling framework developed in Bhattacharya (2021a) and Bhattacharya (2021b), and illustrate the methodology with simulation experiments comprising a two-dimensional normal-gamma posterior, a two-dimensional Ising model posterior, a two-dimensional Strauss process posterior and a 100-dimensional autologistic model posterior. In each case we demonstrate great accuracy of our methodology, which is also computationally extremely efficient, often taking only a few minutes for generating 10, 000 iid realizations on 80 processors.