An MCMC hypothesis test to check a claimed sampler: applied to a claimed sampler for the G-Wishart distribution

TL;DR

This paper introduces a hypothesis testing method based on MCMC techniques to verify the correctness of claimed samplers, demonstrated on a disputed G-Wishart sampler, revealing its invalidity.

Contribution

The paper develops a novel MCMC-based hypothesis test for validating samplers and applies it to assess a controversial G-Wishart sampler, providing a practical validation tool.

Findings

The claimed G-Wishart sampler is invalid according to the test.

The test correctly identifies the known exact sampler as valid.

The method offers a statistical approach to verify sampler correctness.

Abstract

Suppose we have a distribution of interest, with density $p(x),x\in {\cal X}$ say, and an algorithm claimed to generate samples from $p(x)$. Moreover, assume we have available a Metropolis--Hastings transition kernel fulfilling detail balance with respect to $p(x)$. In such a situation we formulate a hypothesis test where $H_0$ is that the claimed sampler really generates correct samples from $p(x)$. We use that if initialising the Metropolis--Hastings algorithm with a sample generated by the claimed sampler and run the chain for a fixed number of updates, the initial and final states are exchangeable if $H_0$ is true. Combining this idea with the permutation strategy we define a natural test statistic and a valid p-value. Our motivation for considering the hypothesis test situation is a proposed sampler in the literature, claimed to generate samples from G-Wishart distribution. As no proper proof for the validity of this sampler seems to be available, we are exactly in the hypothesis test situation discussed above. We therefore apply the defined hypothesis test to the claimed sampler. For comparison we also apply the hypothesis test to a known exact sampler for a subset of G-Wishart distributions. The obtained p-values clearly show that the sampler claimed to be able to generate samples from any G-Wishart distribution is in fact not sampling from the specified distribution. In contrast, and as one should expect, the p-values obtained when using the known exact algorithm does not indicate any problems.