A Comparative Study of UMAP and Other Dimensionality Reduction Methods

TL;DR

This paper compares UMAP with other dimensionality reduction methods, evaluating their performance on classification and regression tasks using simulated and real datasets.

Contribution

It provides a systematic evaluation of supervised UMAP, revealing its strengths in classification and limitations in regression applications.

Findings

Supervised UMAP performs well for classification tasks.

Supervised UMAP has limitations in regression settings.

The study offers insights into the effectiveness of various dimensionality reduction methods.

Abstract

Uniform Manifold Approximation and Projection (UMAP) is a widely used manifold learning technique for dimensionality reduction. This paper studies UMAP, supervised UMAP, and several competing dimensionality reduction methods, including Principal Component Analysis (PCA), Kernel PCA, Sliced Inverse Regression (SIR), Kernel SIR, and t-distributed Stochastic Neighbor Embedding, through a comprehensive comparative analysis. Although UMAP has attracted substantial attention for preserving local and global structures, its supervised extensions, particularly for regression settings, remain rather underexplored. We provide a systematic evaluation of supervised UMAP for both regression and classification using simulated and real datasets, with performance assessed via predictive accuracy on low-dimensional embeddings. Our results show that supervised UMAP performs well for classification but exhibits limitations in effectively incorporating response information for regression, highlighting an important direction for future development.