Composite Bayesian inference

TL;DR

This paper introduces composite Bayesian inference, a method that combines multiple Bayesian agents to improve probabilistic predictions, balancing interpretability and performance.

Contribution

It generalizes composite likelihood to a broader class of models, providing a way to optimize weights either a priori or through learning to enhance predictive accuracy.

Findings

Weights can be optimized via convex cross-entropy minimization.

Composite Bayesian offers a middle ground between generative and discriminative models.

The approach improves prediction performance while maintaining interpretability.

Abstract

We revisit and generalize the concept of composite likelihood as a method to make a probabilistic inference by aggregation of multiple Bayesian agents, thereby defining a class of predictive models which we call composite Bayesian. This perspective gives insight to choose the weights associated with composite likelihood, either a priori or via learning; in the latter case, they may be tuned so as to minimize prediction cross-entropy, yielding an easy-to-solve convex problem. We argue that composite Bayesian inference is a middle way between generative and discriminative models that trades off between interpretability and prediction performance, both of which are crucial to many artificial intelligence tasks.