SMML estimators for exponential families with continuous sufficient statistics

TL;DR

This paper investigates the structure of SMML estimators for exponential families with continuous sufficient statistics, providing a new proof approach and explicit construction methods for specific cases.

Contribution

It offers a novel proof using calculus of variations for the structure of SMML estimators and constructs an explicit estimator for a 2D normal model.

Findings

SMML estimators partition parameter space into convex polytopes

New inequalities for SMML estimators derived from the proof

Explicit construction of SMML estimator for 2D normal with known variance

Abstract

The minimum message length principle is an information theoretic criterion that links data compression with statistical inference. This paper studies the strict minimum message length (SMML) estimator for $d$-dimensional exponential families with continuous sufficient statistics, for all $d \ge 1$. The partition of an SMML estimator is shown to consist of convex polytopes (i.e. convex polygons when $d=2$) which can be described explicitly in terms of the assertions and coding probabilities. While this result is known, we give a new proof based on the calculus of variations, and this approach gives some interesting new inequalities for SMML estimators. We also use this result to construct an SMML estimator for a $2$-dimensional normal random variable with known variance and a normal prior on its mean.