On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances

TL;DR

This paper introduces the Orlicz-Wasserstein distance to analyze the convergence of Gaussian mixture models, revealing faster contraction rates in outlier regions and providing new theoretical insights and computational methods.

Contribution

The paper proposes a novel Orlicz-Wasserstein metric for Bayesian contraction analysis and demonstrates its effectiveness through theoretical results and a practical Sinkhorn divergence-based algorithm.

Findings

Posterior contraction rates are nearly polynomial in outlier regions.

The new metric offers improved theoretical understanding of parameter convergence.

An efficient algorithm for computing the metric is developed and validated.

Abstract

Dirichlet Process mixture models (DPMM) in combination with Gaussian kernels have been an important modeling tool for numerous data domains arising from biological, physical, and social sciences. However, this versatility in applications does not extend to strong theoretical guarantees for the underlying parameter estimates, for which only a logarithmic rate is achieved. In this work, we (re)introduce and investigate a metric, named Orlicz-Wasserstein distance, in the study of the Bayesian contraction behavior for the parameters. We show that despite the overall slow convergence guarantees for all the parameters, posterior contraction for parameters happens at almost polynomial rates in outlier regions of the parameter space. Our theoretical results provide new insight in understanding the convergence behavior of parameters arising from various settings of hierarchical Bayesian nonparametric models. In addition, we provide an algorithm to compute the metric by leveraging Sinkhorn divergences and validate our findings through a simulation study.