Efficient Uncertainty Propagation in Bayesian Two-Step Procedures

TL;DR

This paper introduces an efficient method for propagating uncertainty in Bayesian two-step procedures, reducing computational costs while accurately capturing both aleatoric and epistemic uncertainties in surrogate modeling and missing data scenarios.

Contribution

It proposes a novel approach using mixture distributions and importance sampling to efficiently approximate posteriors in two-step Bayesian inference tasks.

Findings

Reduces computational overhead compared to traditional methods.

Maintains high accuracy in uncertainty propagation.

Effective in surrogate modeling and missing data applications.

Abstract

Bayesian inference provides a principled framework for probabilistic reasoning. If inference is performed in two steps, uncertainty propagation plays a crucial role in accounting for all sources of uncertainty and variability. This becomes particularly important when both aleatoric uncertainty, caused by data variability, and epistemic uncertainty, arising from incomplete knowledge or missing data, are present. Examples include surrogate models and missing data problems. In surrogate modeling, the surrogate is used as a simplified approximation of a resource-heavy and costly simulation. The uncertainty from the surrogate-fitting process can be propagated using a two-step procedure. For modeling with missing data, methods like Multivariate Imputation by Chained Equations (MICE) generate multiple datasets to account for imputation uncertainty. These approaches, however, are computationally expensive, as multiple models must be fitted separately to surrogate parameters respectively imputed datasets. To address these challenges, we propose an efficient two-step approach that reduces computational overhead while maintaining accuracy. By selecting a representative subset of draws or imputations, we construct a mixture distribution to approximate the desired posteriors using Pareto smoothed importance sampling. For more complex scenarios, this is further refined with importance weighted moment matching and an iterative procedure that broadens the mixture distribution to better capture diverse posterior distributions.