Estimation of Gaussian mixtures in small sample studies using $l_1$ penalization

TL;DR

This paper introduces a robust penalized EM algorithm for estimating Gaussian mixture components from small datasets, outperforming standard maximum likelihood methods in experiments.

Contribution

It develops a novel $l_1$ penalized EM approach with proven convergence properties for small sample Gaussian mixture estimation.

Findings

The proposed method outperforms maximum likelihood estimation in small sample scenarios.

The penalized EM algorithm converges to solutions satisfying KKT conditions.

Monte Carlo experiments validate the robustness and effectiveness of the new estimator.

Abstract

Many experiments in medicine and ecology can be conveniently modeled by finite Gaussian mixtures but face the problem of dealing with small data sets. We propose a robust version of the estimator based on self-regression and sparsity promoting penalization in order to estimate the components of Gaussian mixtures in such contexts. A space alternating version of the penalized EM algorithm is obtained and we prove that its cluster points satisfy the Karush-Kuhn-Tucker conditions. Monte Carlo experiments are presented in order to compare the results obtained by our method and by standard maximum likelihood estimation. In particular, our estimator is seen to perform better than the maximum likelihood estimator.