Dimensionality Reduction Ensembles

TL;DR

This paper introduces a novel ensemble approach for dimensionality reduction combining PCA and manifold learning techniques, demonstrating improved accuracy on datasets with potential for broader application in unsupervised learning.

Contribution

It is the first to systematically explore ensemble methods in dimensionality reduction, integrating multiple techniques to capture diverse data features.

Findings

Ensemble methods improve dimensionality reduction accuracy.

Approach performs well on simulated and real datasets.

Computational cost is a limitation, but can be mitigated.

Abstract

Ensemble learning has had many successes in supervised learning, but it has been rare in unsupervised learning and dimensionality reduction. This study explores dimensionality reduction ensembles, using principal component analysis and manifold learning techniques to capture linear, nonlinear, local, and global features in the original dataset. Dimensionality reduction ensembles are tested first on simulation data and then on two real medical datasets using random forest classifiers; results suggest the efficacy of this approach, with accuracies approaching that of the full dataset. Limitations include computational cost of some algorithms with strong performance, which may be ameliorated through distributed computing and the development of more efficient versions of these algorithms.