On The Gaussian Approximation To Bayesian Posterior Distributions

TL;DR

This paper investigates the conditions under which Bayesian posterior distributions can be approximated as Gaussian, deriving the minimal number of observations needed for such an approximation across different models.

Contribution

It provides a theoretical derivation of the minimal sample size required for Gaussian approximation of Bayesian posteriors, with practical examples involving chi-squared and binomial distributions.

Findings

High minimal N for chi-squared distribution

Low minimal N for binomial distribution

The measure μ on the parameter scale is crucial

Abstract

The present article derives the minimal number $N$ of observations needed to consider a Bayesian posterior distribution as Gaussian. Two examples are presented. Within one of them, a chi-squared distribution, the observable $x$ as well as the parameter $\xi$ are defined all over the real axis, in the other one, the binomial distribution, the observable $x$ is an entire number while the parameter $\xi$ is defined on a finite interval of the real axis. The required minimal $N$ is high in the first case and low for the binomial model. In both cases the precise definition of the measure $\mu$ on the scale of $\xi$ is crucial.