Loss-calibrated expectation propagation for approximate Bayesian decision-making

TL;DR

This paper introduces Loss-EP, a loss-calibrated version of expectation propagation, which improves Bayesian inference by focusing on utility-sensitive posterior approximations for decision-making tasks.

Contribution

The paper develops Loss-EP, integrating utility considerations into expectation propagation to enhance approximate Bayesian inference for decision-making applications.

Findings

Loss-EP effectively tilts the posterior towards higher-utility decisions.

Application to Gaussian process classification shows utility-sensitive approximations.

Asymmetry in utility functions significantly impacts the information captured by the approximation.

Abstract

Approximate Bayesian inference methods provide a powerful suite of tools for finding approximations to intractable posterior distributions. However, machine learning applications typically involve selecting actions, which -- in a Bayesian setting -- depend on the posterior distribution only via its contribution to expected utility. A growing body of work on loss-calibrated approximate inference methods has therefore sought to develop posterior approximations sensitive to the influence of the utility function. Here we introduce loss-calibrated expectation propagation (Loss-EP), a loss-calibrated variant of expectation propagation. This method resembles standard EP with an additional factor that "tilts" the posterior towards higher-utility decisions. We show applications to Gaussian process classification under binary utility functions with asymmetric penalties on False Negative and False Positive errors, and show how this asymmetry can have dramatic consequences on what information is "useful" to capture in an approximation.