Mean-field theory of Bayesian clustering

TL;DR

This paper maps Bayesian clustering to a statistical physics problem, enabling a mean-field analysis that leads to an effective algorithm for inferring the number of clusters and their partitions.

Contribution

It introduces a novel statistical physics framework for Bayesian clustering, providing a new analytical approach and an effective clustering algorithm.

Findings

Mean-field analysis of the entropy function is exact in certain regimes.

The lowest entropy state corresponds to the optimal clustering.

The proposed algorithm infers the number of clusters and their partitions effectively.

Abstract

We show that model-based Bayesian clustering, the probabilistically most systematic approach to the partitioning of data, can be mapped into a statistical physics problem for a gas of particles, and as a result becomes amenable to a detailed quantitative analysis. A central role in the resulting statistical physics framework is played by an entropy function. We demonstrate that there is a relevant parameter regime where mean-field analysis of this function is exact, and that, under natural assumptions, the lowest entropy state of the hypothetical gas corresponds to the optimal clustering of data. The byproduct of our analysis is a simple but effective clustering algorithm, which infers both the most plausible number of clusters in the data and the corresponding partitions. Describing Bayesian clustering in statistical mechanical terms is found to be natural and surprisingly effective.