Dataset-Adaptive Dimensionality Reduction

TL;DR

This paper introduces a dataset-adaptive method for optimizing dimensionality reduction by using structural complexity metrics to predict the necessary embedding dimensions, reducing trial-and-error and computational costs.

Contribution

It proposes a novel set of structural complexity metrics that guide the selection of optimal DR techniques and hyperparameters based on dataset complexity.

Findings

Metrics accurately approximate dataset complexity

The adaptive workflow reduces optimization trials

Efficiency improves without loss of accuracy

Abstract

Selecting the appropriate dimensionality reduction (DR) technique and determining its optimal hyperparameter settings that maximize the accuracy of the output projections typically involves extensive trial and error, often resulting in unnecessary computational overhead. To address this challenge, we propose a dataset-adaptive approach to DR optimization guided by structural complexity metrics. These metrics quantify the intrinsic complexity of a dataset, predicting whether higher-dimensional spaces are necessary to represent it accurately. Since complex datasets are often inaccurately represented in two-dimensional projections, leveraging these metrics enables us to predict the maximum achievable accuracy of DR techniques for a given dataset, eliminating redundant trials in optimizing DR. We introduce the design and theoretical foundations of these structural complexity metrics. We quantitatively verify that our metrics effectively approximate the ground truth complexity of datasets and confirm their suitability for guiding dataset-adaptive DR workflow. Finally, we empirically show that our dataset-adaptive workflow significantly enhances the efficiency of DR optimization without compromising accuracy.