Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility

TL;DR

This paper introduces RBHC, a scalable clustering algorithm derived from Bayesian hierarchical clustering using small-variance asymptotics, combining efficiency with Bayesian flexibility.

Contribution

It develops a novel non-probabilistic clustering method from BHC's asymptotic limit, enabling scalable Bayesian hierarchical clustering.

Findings

RBHC achieves high clustering accuracy on synthetic and real datasets.

The dissimilarity measure from the asymptotic limit is reducible, enabling scalable algorithms.

Numerical experiments validate the method's effectiveness and efficiency.

Abstract

Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluation of marginal likelihoods and careful hyperparameter tuning are cumbersome and limit the scalability. In this paper we relax BHC into a non-probabilistic formulation, exploring small-variance asymptotics in conjugate-exponential models. We develop a novel clustering algorithm, referred to as relaxed BHC (RBHC), from the asymptotic limit of the BHC model that exhibits the scalability of distance-based agglomerative clustering algorithms as well as the flexibility of Bayesian nonparametric models. We also investigate the reducibility of the dissimilarity measure emerged from the asymptotic limit of the BHC model, allowing us to use scalable algorithms such as the nearest neighbor chain algorithm. Numerical experiments on both synthetic and real-world datasets demonstrate the validity and high performance of our method.