Improved information criteria for Bayesian model averaging in lattice field theory

TL;DR

This paper introduces and compares new information criteria derived from the Kullback-Leibler divergence for Bayesian model averaging, demonstrating their effectiveness in lattice field theory calculations especially with noisy data.

Contribution

It revisits the derivation of model weight estimators and proposes alternative information criteria, including BAIC, BPIC, and PPIC, for improved Bayesian model averaging.

Findings

PPIC performs best with noisy data.

BAIC is a simple and reliable alternative.

New criteria have favorable theoretical properties.

Abstract

Bayesian model averaging is a practical method for dealing with uncertainty due to model specification. Use of this technique requires the estimation of model probability weights. In this work, we revisit the derivation of estimators for these model weights. Use of the Kullback-Leibler divergence as a starting point leads naturally to a number of alternative information criteria suitable for Bayesian model weight estimation. We explore three such criteria, known to the statistics literature before, in detail: a Bayesian analogue of the Akaike information criterion which we call the BAIC, the Bayesian predictive information criterion (BPIC), and the posterior predictive information criterion (PPIC). We compare the use of these information criteria in numerical analysis problems common in lattice field theory calculations. We find that the PPIC has the most appealing theoretical properties and can give the best performance in terms of model-averaging uncertainty, particularly in the presence of noisy data, while the BAIC is a simple and reliable alternative.