CBMAP: Clustering-based manifold approximation and projection for dimensionality reduction

TL;DR

CBMAP is a novel clustering-based dimensionality reduction technique that preserves both local and global data structures, offering improved scalability, minimal hyperparameter tuning, and the ability to project test data effectively.

Contribution

This paper introduces CBMAP, a new clustering-based method for dimensionality reduction that overcomes limitations of existing nonlinear techniques by maintaining global and local structures with minimal hyperparameters.

Findings

CBMAP outperforms existing methods in preserving data structures.

CBMAP is faster and more scalable on benchmark datasets.

CBMAP enables effective projection of test data into low-dimensional spaces.

Abstract

Dimensionality reduction methods are employed to decrease data dimensionality, either to enhance machine learning performance or to facilitate data visualization in two or three-dimensional spaces. These methods typically fall into two categories: feature selection and feature transformation. Feature selection retains significant features, while feature transformation projects data into a lower-dimensional space, with linear and nonlinear methods. While nonlinear methods excel in preserving local structures and capturing nonlinear relationships, they may struggle with interpreting global structures and can be computationally intensive. Recent algorithms, such as the t-SNE, UMAP, TriMap, and PaCMAP prioritize preserving local structures, often at the expense of accurately representing global structures, leading to clusters being spread out more in lower-dimensional spaces. Moreover, these methods heavily rely on hyperparameters, making their results sensitive to parameter settings. To address these limitations, this study introduces a clustering-based approach, namely CBMAP (Clustering-Based Manifold Approximation and Projection), for dimensionality reduction. CBMAP aims to preserve both global and local structures, ensuring that clusters in lower-dimensional spaces closely resemble those in high-dimensional spaces. Experimental evaluations on benchmark datasets demonstrate CBMAP's efficacy, offering speed, scalability, and minimal reliance on hyperparameters. Importantly, CBMAP enables low-dimensional projection of test data, addressing a critical need in machine learning applications. CBMAP is made freely available at https://github.com/doganlab/cbmap and can be installed from the Python Package Directory (PyPI) software repository with the command pip install cbmap.