A general framework for probabilistic model uncertainty

TL;DR

This paper introduces a novel framework for quantifying model uncertainty by treating missing data as the source of uncertainty and using predictive resampling, avoiding subjective priors and standard Bayesian methods.

Contribution

It proposes a new approach that uses sequential predictive densities and missing data to measure model uncertainty, providing an alternative to traditional Bayesian techniques.

Findings

Effective in density estimation tasks

Applicable to variable selection problems

Addresses issues with prior specification

Abstract

Existing approaches to model uncertainty typically either compare models using a quantitative model selection criterion or evaluate posterior model probabilities having set a prior. In this paper, we propose an alternative strategy which views missing observations as the source of model uncertainty, where the true model would be identified with the complete data. To quantify model uncertainty, it is then necessary to provide a probability distribution for the missing observations conditional on what has been observed. This can be set sequentially using one-step-ahead predictive densities, which recursively sample from the best model according to some consistent model selection criterion. Repeated predictive sampling of the missing data, to give a complete dataset and hence a best model each time, provides our measure of model uncertainty. This approach bypasses the need for subjective prior specification or integration over parameter spaces, addressing issues with standard methods such as the Bayes factor. Predictive resampling also suggests an alternative view of hypothesis testing as a decision problem based on a population statistic, where we directly index the probabilities of competing models. In addition to hypothesis testing, we demonstrate our approach on illustrations from density estimation and variable selection.