Multi-view Sparse Laplacian Eigenmaps for nonlinear Spectral Feature Selection

TL;DR

This paper introduces Multi-view Sparse Laplacian Eigenmaps (MSLE), a graph-based feature selection method that combines multiple data views and enforces sparsity to effectively reduce high-dimensional data while preserving its structure.

Contribution

The paper proposes a novel MSLE method that integrates multi-view data, sparsity constraints, and scalable optimization for effective nonlinear spectral feature selection.

Findings

MSLE achieves up to 90% feature reduction with minimal error increase.

SVM maintains 2.72% error rate with 90% feature reduction.

80% feature reduction yields 96.69% accuracy on UCI-HAR dataset.

Abstract

The complexity of high-dimensional datasets presents significant challenges for machine learning models, including overfitting, computational complexity, and difficulties in interpreting results. To address these challenges, it is essential to identify an informative subset of features that captures the essential structure of the data. In this study, the authors propose Multi-view Sparse Laplacian Eigenmaps (MSLE) for feature selection, which effectively combines multiple views of the data, enforces sparsity constraints, and employs a scalable optimization algorithm to identify a subset of features that capture the fundamental data structure. MSLE is a graph-based approach that leverages multiple views of the data to construct a more robust and informative representation of high-dimensional data. The method applies sparse eigendecomposition to reduce the dimensionality of the data, yielding a reduced feature set. The optimization problem is solved using an iterative algorithm alternating between updating the sparse coefficients and the Laplacian graph matrix. The sparse coefficients are updated using a soft-thresholding operator, while the graph Laplacian matrix is updated using the normalized graph Laplacian. To evaluate the performance of the MSLE technique, the authors conducted experiments on the UCI-HAR dataset, which comprises 561 features, and reduced the feature space by 10 to 90%. Our results demonstrate that even after reducing the feature space by 90%, the Support Vector Machine (SVM) maintains an error rate of 2.72%. Moreover, the authors observe that the SVM exhibits an accuracy of 96.69% with an 80% reduction in the overall feature space.