A Survey: Potential Dimensionality Reduction Methods

TL;DR

This survey compares five key dimensionality reduction techniques, discussing their mathematical foundations, strengths, limitations, and suitability for various data analysis and visualization tasks.

Contribution

It provides a comprehensive overview of popular dimensionality reduction methods, highlighting their differences and guiding practitioners in method selection.

Findings

PCA captures linear variance efficiently.

t-SNE and UMAP excel in local structure preservation.

Trade-offs exist between global structure and computational efficiency.

Abstract

Dimensionality reduction is a fundamental technique in machine learning and data analysis, enabling efficient representation and visualization of high-dimensional data. This paper explores five key methods: Principal Component Analysis (PCA), Kernel PCA (KPCA), Sparse Kernel PCA, t-Distributed Stochastic Neighbor Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP). PCA provides a linear approach to capturing variance, whereas KPCA and Sparse KPCA extend this concept to non-linear structures using kernel functions. Meanwhile, t-SNE and UMAP focus on preserving local relationships, making them effective for data visualization. Each method is examined in terms of its mathematical formulation, computational complexity, strengths, and limitations. The trade-offs between global structure preservation, computational efficiency, and interpretability are discussed to guide practitioners in selecting the appropriate technique based on their application needs.