A Bayesian approach to learning mixtures of nonparametric components

TL;DR

This paper introduces a Bayesian nonparametric approach for learning mixtures with complex, nonparametric components, providing theoretical guarantees and an efficient inference algorithm, advancing the modeling of heterogeneous data.

Contribution

It develops a Bayesian framework for finite mixtures with nonparametric components, including conditions for identifiability and posterior contraction analysis.

Findings

Posterior contraction rate is nearly polynomial.

Efficient MCMC algorithm for inference.

Successful application to real-world data.

Abstract

Mixture models are widely used in modeling heterogeneous data populations. A standard approach of mixture modeling assumes that the mixture component takes a parametric kernel form. In many applications, making parametric assumptions on the latent subpopulation distributions may be unrealistic, which motivates the need for nonparametric modeling of the mixture components themselves. In this paper, we study finite mixtures with nonparametric mixture components, using a Bayesian nonparametric modeling approach. In particular, it is assumed that the data population is generated according to a finite mixture of latent component distributions, where each component is endowed with a Bayesian nonparametric prior such as the Dirichlet process mixture. We present conditions under which the individual mixture component's distribution can be identified, and establish posterior contraction behavior for the data population's density, as well as densities of the latent mixture components. We develop an efficient MCMC algorithm for posterior inference and demonstrate via simulation studies and real-world data illustrations that it is possible to efficiently learn complex forms of probability distribution for the latent subpopulations. In theory, the posterior contraction rate of the component densities is nearly polynomial, which is a significant improvement over the logarithmic convergence rates of estimating mixing measures via deconvolution.