Replica analysis of Bayesian data clustering

TL;DR

This paper applies statistical mechanics and the replica method to analyze Bayesian data clustering, providing a theoretical framework that accurately predicts clustering behavior and improves upon previous mean-field approaches.

Contribution

It introduces a replica symmetric theory for Bayesian clustering, deriving self-consistent equations for the order parameter and demonstrating improved predictive accuracy over earlier models.

Findings

The theory matches numerical experiments perfectly.

It offers a significant improvement over previous mean-field models.

The population dynamics algorithm effectively solves the self-consistent equations.

Abstract

We use statistical mechanics to study model-based Bayesian data clustering. In this approach, each partition of the data into clusters is regarded as a microscopic system state, the negative data log-likelihood gives the energy of each state, and the data set realisation acts as disorder. Optimal clustering corresponds to the ground state of the system, and is hence obtained from the free energy via a low `temperature' limit. We assume that for large sample sizes the free energy density is self-averaging, and we use the replica method to compute the asymptotic free energy density. The main order parameter in the resulting (replica symmetric) theory, the distribution of the data over the clusters, satisfies a self-consistent equation which can be solved by a population dynamics algorithm. From this order parameter one computes the average free energy, and all relevant macroscopic characteristics of the problem. The theory describes numerical experiments perfectly, and gives a significant improvement over the mean-field theory that was used to study this model in past.