Correcting Predictions for Approximate Bayesian Inference

TL;DR

This paper introduces a method to correct decision-making errors caused by approximate Bayesian inference by training a model to optimize predictive accuracy, applicable across various probabilistic models.

Contribution

It presents a novel approach that adjusts decision-making under approximate posteriors, improving accuracy without changing the inference method itself.

Findings

Effective correction of approximate inference errors

Applicable to arbitrary probabilistic programs

Improved decision quality demonstrated empirically

Abstract

Bayesian models quantify uncertainty and facilitate optimal decision-making in downstream applications. For most models, however, practitioners are forced to use approximate inference techniques that lead to sub-optimal decisions due to incorrect posterior predictive distributions. We present a novel approach that corrects for inaccuracies in posterior inference by altering the decision-making process. We train a separate model to make optimal decisions under the approximate posterior, combining interpretable Bayesian modeling with optimization of direct predictive accuracy in a principled fashion. The solution is generally applicable as a plug-in module for predictive decision-making for arbitrary probabilistic programs, irrespective of the posterior inference strategy. We demonstrate the approach empirically in several problems, confirming its potential.