Besag-Clifford e-values for unnormalized testing

TL;DR

This paper introduces Besag-Clifford e-values for valid hypothesis testing using unnormalized distributions, leveraging exchangeable sampling to handle intractable normalizing constants.

Contribution

It proposes a novel method to generate valid e-values from unnormalized likelihoods, extending to composite hypotheses and sequential testing.

Findings

Besag-Clifford e-values are asymptotically log-optimal up to a diminishing factor.

Averaging over multiple chains increases e-power while maintaining validity.

Empirical results confirm the theoretical properties and practical effectiveness.

Abstract

Unnormalized probability distributions are frequently used in machine learning for modeling complex data generating processes. Though Markov chain Monte Carlo (MCMC) algorithms can approximately sample from unnormalized distributions, intractability of their normalizing constants renders likelihood ratio testing infeasible. We propose to use the parallel method of Besag and Clifford to generate samples that are exchangeable with the data under the null, to then generate valid e-values for any number of iterations or algorithmic steps. We show that as the number of samples grows, these Besag-Clifford e-values constructed using the unnormalized likelihood ratio are actually log-optimal up to a multiplicative term that diminishes with the mixing time of the Markov chain. Additionally, averaging over the output of multiple chains retains validity while increasing the e-power. We extend Besag-Clifford e-values to the general problem of unnormalized test statistics, which allows application to composite hypotheses, uncertainty quantification, generative model evaluation, and sequential testing. Through simulations and an application to galaxy velocity modeling, we empirically verify our theory, explore the impact of autocorrelation and mixing, and evaluate the performance of Besag-Clifford e-values.