On the cohesion and separability of average-link for hierarchical agglomerative clustering

TL;DR

This paper provides a comprehensive theoretical and experimental analysis of average-link hierarchical clustering, demonstrating its advantages in metric spaces concerning cohesion and separability over other methods.

Contribution

It offers new theoretical insights and experimental evidence showing average-link's superior performance in metric spaces for cohesion and separability criteria.

Findings

Average-link outperforms other methods in metric spaces for cohesion and separability.

Theoretical analyses show average-link's better approximation properties.

Experimental results support average-link as a preferable choice for real datasets.

Abstract

Average-link is widely recognized as one of the most popular and effective methods for building hierarchical agglomerative clustering. The available theoretical analyses show that this method has a much better approximation than other popular heuristics, as single-linkage and complete-linkage, regarding variants of Dasgupta's cost function [STOC 2016]. However, these analyses do not separate average-link from a random hierarchy and they are not appealing for metric spaces since every hierarchical clustering has a 1/2 approximation with regard to the variant of Dasgupta's function that is employed for dissimilarity measures [Moseley and Yang 2020]. In this paper, we present a comprehensive study of the performance of average-link in metric spaces, regarding several natural criteria that capture separability and cohesion and are more interpretable than Dasgupta's cost function and its variants. We also present experimental results with real datasets that, together with our theoretical analyses, suggest that average-link is a better choice than other related methods when both cohesion and separability are important goals.