Modifications of the BIC for order selection in finite mixture models

TL;DR

This paper proposes modified BIC criteria for finite mixture model order selection that are consistent under weaker assumptions, including non-differentiability and mild moment conditions, and discusses their limitations.

Contribution

Introduction of the $ u$-BIC and $\e$-BIC, which improve consistency in mixture model order selection under less restrictive conditions.

Findings

Modified BIC criteria are consistent with weaker assumptions.

Misspecification results show optimal order selection outside the candidate family.

Limitations include no universal minimal penalty and potential conflict with minimax optimality.

Abstract

Finite mixture models are ubiquitous in modern statistical modeling, and a recurring practical issue is choosing the model order. In \citet[Sankhy\=a Series A, \textbf62, pp. 49--66]{keribin2000consistent}, the Bayesian information criterion (BIC) was proved consistent in mixtures, but under strong regularity, including high moments and high-order derivatives of the component density. We introduce the $\nu$-BIC and $\epsilon$-BIC, which weight the BIC penalty by negligibly small logarithmic factors immaterial in practice. This minor modification yields consistency under substantially weaker conditions, without differentiability and with mild moment assumptions, and we also give a misspecification result: when the truth lies outside the candidate family, any vanishing-penalty IC eventually selects a Kullback--Leibler optimal order among candidates. Finally, we clarify two limitations of consistent IC-based selection in mixtures: there is no universally minimal BIC-scale penalty within our sufficient conditions, and order consistency can conflict with minimax optimality in Hellinger risk. We illustrate the theory for Gaussian mixtures, non-differentiable Laplace mixtures, heavy-tailed $t$-mixtures, and mixtures of regression models.