Comparing Euclidean and Hyperbolic K-Means for Generalized Category Discovery

TL;DR

This paper introduces HC-GCD, a hyperbolic embedding and clustering method for generalized category discovery, demonstrating improved clustering accuracy and consistency over Euclidean approaches by directly operating in hyperbolic space.

Contribution

It proposes learning hyperbolic embeddings in the Lorentz model and clustering directly in hyperbolic space, advancing hyperbolic GCD methods beyond previous Euclidean transformations.

Findings

HC-GCD matches state-of-the-art hyperbolic GCD performance.

Hyperbolic K-Means outperforms Euclidean K-Means in accuracy.

Hyperbolic clustering yields more consistent results across label granularities.

Abstract

Hyperbolic representation learning has been widely used to extract implicit hierarchies within data, and recently it has found its way to the open-world classification task of Generalized Category Discovery (GCD). However, prior hyperbolic GCD methods only use hyperbolic geometry for representation learning and transform back to Euclidean geometry when clustering. We hypothesize this is suboptimal. Therefore, we present Hyperbolic Clustered GCD (HC-GCD), which learns embeddings in the Lorentz Hyperboloid model of hyperbolic geometry, and clusters these embeddings directly in hyperbolic space using a hyperbolic K-Means algorithm. We test our model on the Semantic Shift Benchmark datasets, and demonstrate that HC-GCD is on par with the previous state-of-the-art hyperbolic GCD method. Furthermore, we show that using hyperbolic K-Means leads to better accuracy than Euclidean K-Means. We carry out ablation studies showing that clipping the norm of the Euclidean embeddings leads to decreased accuracy in clustering unseen classes, and increased accuracy for seen classes, while the overall accuracy is dataset dependent. We also show that using hyperbolic K-Means leads to more consistent clusters when varying the label granularity.