HSC: A Novel Method for Clustering Hierarchies of Networked Data

TL;DR

HSC introduces a two-stage hierarchical clustering framework that improves cluster quality and computational efficiency for large datasets by combining primary hierarchy construction with detailed clustering refinement.

Contribution

The paper proposes a novel divisive hierarchical clustering framework called HSC, integrating stochastic clustering and variable aggregation theories for better hierarchical data organization.

Findings

Builds meaningful cluster hierarchies

Enhances clustering quality in second-stage algorithms

Improves computational efficiency for large datasets

Abstract

Hierarchical clustering is one of the most powerful solutions to the problem of clustering, on the grounds that it performs a multi scale organization of the data. In recent years, research on hierarchical clustering methods has attracted considerable interest due to the demanding modern application domains. We present a novel divisive hierarchical clustering framework called Hierarchical Stochastic Clustering (HSC), that acts in two stages. In the first stage, it finds a primary hierarchy of clustering partitions in a dataset. In the second stage, feeds a clustering algorithm with each one of the clusters of the very detailed partition, in order to settle the final result. The output is a hierarchy of clusters. Our method is based on the previous research of Meyer and Weissel Stochastic Data Clustering and the theory of Simon and Ando on Variable Aggregation. Our experiments show that our framework builds a meaningful hierarchy of clusters and benefits consistently the clustering algorithm that acts in the second stage, not only computationally but also in terms of cluster quality. This result suggest that HSC framework is ideal for obtaining hierarchical solutions of large volumes of data.