On choosing mixture components via non-local priors

TL;DR

This paper introduces non-local priors (NLPs) for mixture models, enabling better separation of components and more accurate determination of the number of mixture components, addressing limitations of traditional criteria.

Contribution

It formalizes NLPs for mixtures, develops estimators for posterior model probabilities, and provides theoretical and computational tools for improved mixture component selection.

Findings

NLPs lead to well-separated, interpretable mixture components.

Bayes factors are ratios of posterior to prior empty-cluster probabilities.

Default NLP priors balance sparsity and component detection.

Abstract

Choosing the number of mixture components remains an elusive challenge. Model selection criteria can be either overly liberal or conservative and return poorly-separated components of limited practical use. We formalize non-local priors (NLPs) for mixtures and show how they lead to well-separated components with non-negligible weight, interpretable as distinct subpopulations. We also propose an estimator for posterior model probabilities under local and non-local priors, showing that Bayes factors are ratios of posterior to prior empty-cluster probabilities. The estimator is widely applicable and helps set thresholds to drop unoccupied components in overfitted mixtures. We suggest default prior parameters based on multi-modality for Normal/T mixtures and minimal informativeness for categorical outcomes. We characterise theoretically the NLP-induced sparsity, derive tractable expressions and algorithms. We fully develop Normal, Binomial and product Binomial mixtures but the theory, computation and principles hold more generally. We observed a serious lack of sensitivity of the Bayesian information criterion (BIC), insufficient parsimony of the AIC and a local prior, and a mixed behavior of the singular BIC. We also considered overfitted mixtures, their performance was competitive but depended on tuning parameters. Under our default prior elicitation NLPs offered a good compromise between sparsity and power to detect meaningfully-separated components.