Improving Inference of Gaussian Mixtures Using Auxiliary Variables

TL;DR

This paper provides a theoretical foundation for improving Gaussian mixture inference by incorporating auxiliary variables, demonstrating enhanced accuracy in membership allocation and parameter estimation through analytical results, simulations, and real-world applications.

Contribution

It offers the first theoretical justification for using auxiliary variables in mixture models, showing their benefits over univariate models in various scenarios.

Findings

Auxiliary variables improve mixture component allocation accuracy.

Inclusion of auxiliary variables increases information about mixture means.

Simulation results confirm robustness even with model misspecification.

Abstract

Expanding a lower-dimensional problem to a higher-dimensional space and then projecting back is often beneficial. This article rigorously investigates this perspective in the context of finite mixture models, namely how to improve inference for mixture models by using auxiliary variables. Despite the large literature in mixture models and several empirical examples, there is no previous work that gives general theoretical justification for including auxiliary variables in mixture models, even for special cases. We provide a theoretical basis for comparing inference for mixture multivariate models with the corresponding inference for marginal univariate mixture models. Analytical results for several special cases are established. We show that the probability of correctly allocating mixture memberships and the information number for the means of the primary outcome in a bivariate model with two Gaussian mixtures are generally larger than those in each univariate model. Simulations under a range of scenarios, including misspecified models, are conducted to examine the improvement. The method is illustrated by two real applications in ecology and causal inference.