Agglomerative Info-Clustering

TL;DR

This paper introduces an agglomerative info-clustering method for random variables that merges clusters based on maximum multivariate mutual information, enabling earlier stopping and improved efficiency.

Contribution

It presents a novel agglomerative clustering algorithm for info-clustering that leverages submodularity and duality, offering computational advantages over previous methods.

Findings

Efficient algorithm derived using submodularity of entropy

Allows earlier stopping when desired cluster size and accuracy are reached

Improves computational efficiency over existing info-clustering methods

Abstract

An agglomerative clustering of random variables is proposed, where clusters of random variables sharing the maximum amount of multivariate mutual information are merged successively to form larger clusters. Compared to the previous info-clustering algorithms, the agglomerative approach allows the computation to stop earlier when clusters of desired size and accuracy are obtained. An efficient algorithm is also derived based on the submodularity of entropy and the duality between the principal sequence of partitions and the principal sequence for submodular functions.