Minimum message length inference of the Poisson and geometric models using heavy-tailed prior distributions

TL;DR

This paper applies the minimum message length principle with heavy-tailed priors to improve model selection for Poisson and geometric distributions in small samples, outperforming other Bayesian and MDL methods.

Contribution

It introduces heavy-tailed prior distributions for MML inference in Poisson and geometric models, enhancing model selection accuracy in small-sample scenarios.

Findings

MML with heavy-tailed priors performs best across tests.

Heavy-tailed priors improve small-sample model selection.

Compared methods include objective Bayesian and MDL techniques.

Abstract

Minimum message length is a general Bayesian principle for model selection and parameter estimation that is based on information theory. This paper applies the minimum message length principle to a small-sample model selection problem involving Poisson and geometric data models. Since MML is a Bayesian principle, it requires prior distributions for all model parameters. We introduce three candidate prior distributions for the unknown model parameters with both light- and heavy-tails. The performance of the MML methods is compared with objective Bayesian inference and minimum description length techniques based on the normalized maximum likelihood code. Simulations show that our MML approach with a heavy-tail prior distribution provides an excellent performance in all tests.