Variational Learning of Fractional Posteriors

TL;DR

This paper introduces a new variational objective for estimating fractional posteriors, improving calibration and evidence bounds in probabilistic models like VAEs.

Contribution

It presents a novel variational framework for fractional posteriors, extending to hierarchical models and demonstrating improved calibration and evidence bounds.

Findings

Fractional posteriors achieve better calibration than traditional variational posteriors.

The approach yields higher evidence bounds in variational autoencoders.

VAEs trained with fractional posteriors produce decoders better aligned for prior generation.

Abstract

We introduce a novel one-parameter variational objective that lower bounds the data evidence and enables the estimation of approximate fractional posteriors. We extend this framework to hierarchical construction and Bayes posteriors, offering a versatile tool for probabilistic modelling. We demonstrate two cases where gradients can be obtained analytically and a simulation study on mixture models showing that our fractional posteriors can be used to achieve better calibration compared to posteriors from the conventional variational bound. When applied to variational autoencoders (VAEs), our approach attains higher evidence bounds and enables learning of high-performing approximate Bayes posteriors jointly with fractional posteriors. We show that VAEs trained with fractional posteriors produce decoders that are better aligned for generation from the prior.