Adaptive Low-Complexity Sequential Inference for Dirichlet Process Mixture Models

TL;DR

This paper introduces an efficient online inference method for Dirichlet process Gaussian mixture models, enabling adaptive clustering with unknown number of clusters, and demonstrates its superiority through experiments.

Contribution

It presents a novel low-complexity, sequential inference algorithm with adaptive hyperparameter updates for Dirichlet process mixtures, improving scalability and performance.

Findings

Number of clusters grows at most logarithmically with data size.

Conditional likelihood and predictive distribution become Gaussian asymptotically.

Outperforms existing online clustering methods on synthetic and real data.

Abstract

We develop a sequential low-complexity inference procedure for Dirichlet process mixtures of Gaussians for online clustering and parameter estimation when the number of clusters are unknown a-priori. We present an easily computable, closed form parametric expression for the conditional likelihood, in which hyperparameters are recursively updated as a function of the streaming data assuming conjugate priors. Motivated by large-sample asymptotics, we propose a novel adaptive low-complexity design for the Dirichlet process concentration parameter and show that the number of classes grow at most at a logarithmic rate. We further prove that in the large-sample limit, the conditional likelihood and data predictive distribution become asymptotically Gaussian. We demonstrate through experiments on synthetic and real data sets that our approach is superior to other online state-of-the-art methods.