A Unified Maximum Likelihood Approach for Optimal Distribution Property Estimation

TL;DR

This paper demonstrates that a single profile maximum likelihood estimator can optimally estimate various symmetric distribution properties, simplifying the approach and potentially unifying the estimation process.

Contribution

It proves that PML performs as well as specialized methods for multiple distribution properties, suggesting a universal estimation technique.

Findings

PML matches the performance of specialized estimators for support size and entropy.

A single estimator can effectively estimate multiple distribution properties.

Potential for PML to be a universal tool for symmetric property estimation.

Abstract

The advent of data science has spurred interest in estimating properties of distributions over large alphabets. Fundamental symmetric properties such as support size, support coverage, entropy, and proximity to uniformity, received most attention, with each property estimated using a different technique and often intricate analysis tools. We prove that for all these properties, a single, simple, plug-in estimator---profile maximum likelihood (PML)---performs as well as the best specialized techniques. This raises the possibility that PML may optimally estimate many other symmetric properties.