A numerically stable online implementation and exploration of WAIC through variations of the predictive density, using NIMBLE

TL;DR

This paper presents a numerically stable, online algorithm for computing WAIC in NIMBLE, allowing flexible predictive density choices and demonstrating the impact of predictive density forms on WAIC's sensitivity.

Contribution

It introduces a robust online implementation of WAIC that does not require storing all posterior samples and explores the effects of different predictive densities on WAIC calculations.

Findings

WAIC sensitivity depends on the grouping of observations with marginalized predictive densities

The online algorithm improves computational efficiency and stability

Different predictive density choices influence WAIC's predictive accuracy assessment

Abstract

We go through the process of crafting a robust and numerically stable online algorithm for the computation of the Watanabe-Akaike information criteria (WAIC). We implement this algorithm in the NIMBLE software. The implementation is performed in an online manner and does not require the storage in memory of the complete samples from the posterior distribution. This algorithm allows the user to specify a specific form of the predictive density to be used in the computation of WAIC, in order to cater to specific prediction goals. We then comment and explore via simulations the use of different forms of the predictive density in the context of different predictive goals. We find that when using marginalized predictive densities, WAIC is sensitive to the grouping of the observations into a joint density.