CMET: Clustering guided METric for quantifying embedding quality

TL;DR

This paper introduces CMET, a novel clustering-guided metric for efficiently quantifying how well embeddings preserve local and global data structures, demonstrating superior performance over existing methods across diverse datasets.

Contribution

The study proposes CMET, a new, computationally efficient metric for assessing embedding quality, focusing on local and global shape preservation, with broad applicability and stability.

Findings

CMET outperforms state-of-the-art metrics in various datasets.

CMET is computationally efficient and scalable to large datasets.

It provides stable and reliable measurements of embedding quality.

Abstract

Due to rapid advancements in technology, datasets are available from various domains. In order to carry out more relevant and appropriate analysis, it is often necessary to project the dataset into a higher or lower dimensional space based on requirement. Projecting the data in a higher-dimensional space helps in unfolding intricate patterns, enhancing the performance of the underlying models. On the other hand, dimensionality reduction is helpful in denoising data while capturing maximal information, as well as reducing execution time and memory.In this context, it is not always statistically evident whether the transformed embedding retains the local and global structure of the original data. Most of the existing metrics that are used for comparing the local and global shape of the embedding against the original one are highly expensive in terms of time and space complexity. In order to address this issue, the objective of this study is to formulate a novel metric, called Clustering guided METric (CMET), for quantifying embedding quality. It is effective to serve the purpose of quantitative comparison between an embedding and the original data. CMET consists of two scores, viz., CMET_L and CMET_G, that measure the degree of local and global shape preservation capability, respectively. The efficacy of CMET has been demonstrated on a wide variety of datasets, including four synthetic, two biological, and two image datasets. Results reflect the favorable performance of CMET against the state-of-the-art methods. Capability to handle both small and large data, low algorithmic complexity, better and stable performance across all kinds of data, and different choices of hyper-parameters feature CMET as a reliable metric.