Anchored Bayesian Gaussian Mixture Models

TL;DR

This paper introduces an anchored Bayesian Gaussian mixture model that assigns meaningful labels to components during modeling, enabling direct interpretation of component features without post-processing, and provides asymptotic guidelines for model selection.

Contribution

The paper proposes a novel non-exchangeable Bayesian mixture model with data-dependent priors that allows direct inference on component labels and features, improving interpretability.

Findings

Produces interpretable component labels without relabeling.

Aligns with results from relabeling algorithms in many cases.

Offers practical guidelines for model selection based on prior information.

Abstract

Finite mixtures are a flexible modeling tool for irregularly shaped densities and samples from heterogeneous populations. When modeling with mixtures using an exchangeable prior on the component features, the component labels are arbitrary and are indistinguishable in posterior analysis. This makes it impossible to attribute any meaningful interpretation to the marginal posterior distributions of the component features. We propose a model in which a small number of observations are assumed to arise from some of the labeled component densities. The resulting model is not exchangeable, allowing inference on the component features without post-processing. Our method assigns meaning to the component labels at the modeling stage and can be justified as a data-dependent informative prior on the labelings. We show that our method produces interpretable results, often (but not always) similar to those resulting from relabeling algorithms, with the added benefit that the marginal inferences originate directly from a well specified probability model rather than a post hoc manipulation. We provide asymptotic results leading to practical guidelines for model selection that are motivated by maximizing prior information about the class labels and demonstrate our method on real and simulated data.