MML is not consistent for Neyman-Scott

TL;DR

This paper demonstrates that the Strict Minimum Message Length (SMML) method, often believed to be consistent, is actually inconsistent in the Neyman-Scott problem, challenging its assumed reliability in statistical inference.

Contribution

It provides the first explicit construction of an SMML solution for a natural, high-dimensional problem and shows that SMML and its approximations can be inconsistent.

Findings

SMML is inconsistent for Neyman-Scott problem

First explicit construction of SMML solution in high dimensions

Challenges the assumption of SMML's general consistency

Abstract

Strict Minimum Message Length (SMML) is an information-theoretic statistical inference method widely cited (but only with informal arguments) as providing estimations that are consistent for general estimation problems. It is, however, almost invariably intractable to compute, for which reason only approximations of it (known as MML algorithms) are ever used in practice. Using novel techniques that allow for the first time direct, non-approximated analysis of SMML solutions, we investigate the Neyman-Scott estimation problem, an oft-cited showcase for the consistency of MML, and show that even with a natural choice of prior neither SMML nor its popular approximations are consistent for it, thereby providing a counterexample to the general claim. This is the first known explicit construction of an SMML solution for a natural, high-dimensional problem.