Estimating Cross-validatory Predictive P-values with Integrated Importance Sampling for Disease Mapping Models

TL;DR

This paper introduces integrated importance sampling (iIS), a novel efficient method for estimating leave-one-out cross-validatory predictive p-values in disease mapping models, avoiding extensive reruns of MCMC analyses.

Contribution

The paper proposes iIS, a new importance sampling technique that accurately estimates LOOCV predictive p-values using only full data posterior samples, reducing computational cost.

Findings

iIS provides estimates nearly identical to actual LOOCV.

iIS outperforms existing methods like posterior predictive checking, importance sampling, and ghosting.

Empirical tests on disease datasets validate the effectiveness of iIS.

Abstract

An important statistical task in disease mapping problems is to identify divergent regions with unusually high or low risk of disease. Leave-one-out cross-validatory (LOOCV) model assessment is the gold standard for estimating predictive p-values that can flag such divergent regions. However, actual LOOCV is time-consuming because one needs to rerun a Markov chain Monte Carlo analysis for each posterior distribution in which an observation is held out as a test case. This paper introduces a new method, called integrated importance sampling (iIS), for estimating LOOCV predictive p-values with only Markov chain samples drawn from the posterior based on a full data set. The key step in iIS is that we integrate away the latent variables associated the test observation with respect to their conditional distribution \textit{without} reference to the actual observation. By following the general theory for importance sampling, the formula used by iIS can be proved to be equivalent to the LOOCV predictive p-value. We compare iIS and other three existing methods in the literature with two disease mapping datasets. Our empirical results show that the predictive p-values estimated with iIS are almost identical to the predictive p-values estimated with actual LOOCV, and outperform those given by the existing three methods, namely, the posterior predictive checking, the ordinary importance sampling, and the ghosting method by Marshall and Spiegelhalter (2003).