Testing for the Important Components of Posterior Predictive Variance

TL;DR

This paper introduces a method to decompose and test the importance of different components of posterior predictive variance, aiding in identifying key modeling features for prediction accuracy.

Contribution

It provides a novel variance decomposition framework using the law of total variance, with tests to determine the significance of each component in predictive modeling.

Findings

Identifies which features significantly impact prediction intervals.

Offers a criterion for selecting the best variance decomposition for optimal coverage.

Links variance components to interpretability in modeling features.

Abstract

We give a decomposition of the posterior predictive variance using the law of total variance and conditioning on a finite dimensional discrete random variable. This random variable summarizes various features of modeling that are used to form the prediction for a future outcome. Then, we test which terms in this decomposition are small enough to ignore. This allows us identify which of the discrete random variables are most important to prediction intervals. The terms in the decomposition admit interpretations based on conditional means and variances and are analogous to the terms in a Cochran's theorem decomposition of squared error often used in analysis of variance. Thus, the modeling features are treated as factors in completely randomized design. In cases where there are multiple decompositions we suggest choosing the one that that gives the best predictive coverage with the smallest variance.