Multiclass spectral feature scaling method for dimensionality reduction

TL;DR

This paper introduces a spectral feature scaling method for dimensionality reduction that enhances class separation by modifying feature scales based on label information, outperforming existing methods in various datasets.

Contribution

It proposes a novel spectral clustering-based approach to derive feature scaling factors for improved supervised dimensionality reduction.

Findings

Outperforms established methods on toy and real-world datasets.

Effectively separates classes in low-dimensional space.

Handles datasets with more features than samples.

Abstract

Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.