Calibrating Bayesian Generative Machine Learning for Bayesiamplification

TL;DR

This paper proposes a method to quantify the calibration of Bayesian generative models in particle physics, demonstrating how well-calibrated uncertainties can estimate data amplification and improve inference accuracy.

Contribution

It introduces a clear scheme for assessing the calibration of Bayesian generative models, specifically applying it to Continuous Normalizing Flows in a toy example.

Findings

Bayesian uncertainties can be calibrated using the proposed scheme.

Calibrated uncertainties help estimate the effective number of true samples.

Well-calibrated models indicate data amplification for smooth distribution features.

Abstract

Recently, combinations of generative and Bayesian machine learning have been introduced in particle physics for both fast detector simulation and inference tasks. These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics. The interpretation of a distribution-wide uncertainty however remains ill-defined. We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models. For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.