Visual Cluster Separation Using High-Dimensional Sharpened Dimensionality Reduction

TL;DR

This paper introduces HD-SDR, a method that sharpens high-dimensional data clusters before applying dimensionality reduction, resulting in improved 2D visual separation of clusters for exploratory data analysis.

Contribution

The paper presents a novel high-dimensional sharpening step using Local Gradient Clustering prior to DR, enhancing cluster separation in 2D projections.

Findings

HD-SDR improves cluster separation in 2D visualizations.

The method scales well with large high-dimensional datasets.

HD-SDR outperforms DR methods without sharpening in quality metrics.

Abstract

Applying dimensionality reduction (DR) to large, high-dimensional data sets can be challenging when distinguishing the underlying high-dimensional data clusters in a 2D projection for exploratory analysis. We address this problem by first sharpening the clusters in the original high-dimensional data prior to the DR step using Local Gradient Clustering (LGC). We then project the sharpened data from the high-dimensional space to 2D by a user-selected DR method. The sharpening step aids this method to preserve cluster separation in the resulting 2D projection. With our method, end-users can label each distinct cluster to further analyze an otherwise unlabeled data set. Our `High-Dimensional Sharpened DR' (HD-SDR) method, tested on both synthetic and real-world data sets, is favorable to DR methods with poor cluster separation and yields a better visual cluster separation than these DR methods with no sharpening. Our method achieves good quality (measured by quality metrics) and scales computationally well with large high-dimensional data. To illustrate its concrete applications, we further apply HD-SDR on a recent astronomical catalog.