A taxonomy of estimator consistency on discrete estimation problems

TL;DR

This paper introduces a four-level hierarchy classifying discrete estimation problems and estimators based on their consistency guarantees, revealing the scope and limitations of common estimators like MAP, MLE, and SML.

Contribution

It presents a comprehensive hierarchy mapping all discrete estimation problems and estimators, highlighting the consistency guarantees and limitations of popular methods.

Findings

MAP is consistent for the widest class of problems

MLE and Approximate MLE are consistent on a narrower subclass

SML does not guarantee consistency even within its subclass

Abstract

We describe a four-level hierarchy mapping both all discrete estimation problems and all estimators on these problems, such that the hierarchy describes each estimator's consistency guarantees on each problem class. We show that no estimator is consistent for all estimation problems, but that some estimators, such as Maximum A Posteriori, are consistent for the widest possible class of discrete estimation problems. For Maximum Likelihood and Approximate Maximum Likelihood estimators we show that they do not provide consistency on as wide a class, but define a sub-class of problems characterised by their consistency. Lastly, we show that some popular estimators, specifically Strict Minimum Message Length, do not provide consistency guarantees even within the sub-class.