Fast Maximum Likelihood Estimation and Supervised Classification for the Beta-Liouville Multinomial

TL;DR

This paper introduces the Beta-Liouville multinomial distribution, offering a flexible alternative to traditional multinomial models for better parameter estimation and classification accuracy in categorical data analysis.

Contribution

It presents efficient maximum likelihood estimation methods for the Beta-Liouville multinomial and demonstrates its advantages over existing models in classification tasks.

Findings

Beta-Liouville multinomial matches or exceeds Dirichlet multinomial in efficiency.

It outperforms traditional models on two of four datasets.

The distribution is suitable for data with low to medium class overlap.

Abstract

The multinomial and related distributions have long been used to model categorical, count-based data in fields ranging from bioinformatics to natural language processing. Commonly utilized variants include the standard multinomial and the Dirichlet multinomial distributions due to their computational efficiency and straightforward parameter estimation process. However, these distributions make strict assumptions about the mean, variance, and covariance between the categorical features being modeled. If these assumptions are not met by the data, it may result in poor parameter estimates and loss in accuracy for downstream applications like classification. Here, we explore efficient parameter estimation and supervised classification methods using an alternative distribution, called the Beta-Liouville multinomial, which relaxes some of the multinomial assumptions. We show that the Beta-Liouville multinomial is comparable in efficiency to the Dirichlet multinomial for Newton-Raphson maximum likelihood estimation, and that its performance on simulated data matches or exceeds that of the multinomial and Dirichlet multinomial distributions. Finally, we demonstrate that the Beta-Liouville multinomial outperforms the multinomial and Dirichlet multinomial on two out of four gold standard datasets, supporting its use in modeling data with low to medium class overlap in a supervised classification context.