Superiority of Bayes estimators over the MLE in high dimensional multinomial models and its implication for nonparametric Bayes theory

TL;DR

This paper demonstrates that Bayes estimators with Dirichlet priors outperform the maximum likelihood estimator in high-dimensional multinomial models, with implications for nonparametric Bayesian methods.

Contribution

It provides a theoretical comparison showing Bayes estimators' superiority over MLE in high-dimensional multinomial settings, highlighting their practical advantages.

Findings

Bayes estimators outperform MLE in high dimensions

Bayes estimators perform well with moderate sample sizes

Implications for nonparametric Bayesian theory

Abstract

This article focuses on the performance of Bayes estimators, in comparison with the MLE, in multinomial models with a relatively large number of cells. The prior for the Bayes estimator is taken to be the conjugate Dirichlet, i.e., the multivariate Beta, with exchangeable distributions over the coordinates, including the non-informative uniform distribution. The choice of the multinomial is motivated by its many applications in business and industry, but also by its use in providing a simple nonparametric estimator of an unknown distribution. It is striking that the Bayes procedure outperforms the asymptotically efficient MLE over most of the parameter spaces for even moderately large dimensional parameter space and rather large sample sizes.