A data driven equivariant approach to constrained Gaussian mixture modeling

TL;DR

This paper introduces a data-driven, equivariant constrained Gaussian mixture modeling approach that shrinks class-specific covariance matrices towards a chosen matrix to improve maximum likelihood estimation, especially when prior information is lacking.

Contribution

It proposes a novel constrained, equivariant method for Gaussian mixture models that adaptively shrinks covariance matrices, addressing unbounded likelihood issues without requiring strong prior assumptions.

Findings

Effective in avoiding unbounded likelihood problems.

Data-driven selection of shrinkage matrix improves model stability.

Validated through simulation and empirical data.

Abstract

Maximum likelihood estimation of Gaussian mixture models with different class-specific covariance matrices is known to be problematic. This is due to the unboundedness of the likelihood, together with the presence of spurious maximizers. Existing methods to bypass this obstacle are based on the fact that unboundedness is avoided if the eigenvalues of the covariance matrices are bounded away from zero. This can be done imposing some constraints on the covariance matrices, i.e. by incorporating a priori information on the covariance structure of the mixture components. The present work introduces a constrained equivariant approach, where the class conditional covariance matrices are shrunk towards a pre-specified matrix Psi. Data-driven choices of the matrix Psi, when a priori information is not available, and the optimal amount of shrinkage are investigated. The effectiveness of the proposal is evaluated on the basis of a simulation study and an empirical example.