Deep Unsupervised Feature Selection by Discarding Nuisance and Correlated Features

TL;DR

This paper introduces a fully differentiable, autoencoder-based method for unsupervised feature selection that effectively discards nuisance and correlated features, improving clustering performance on real-world datasets.

Contribution

It proposes a novel approach combining Laplacian score criterion with a concrete layer and autoencoder architecture for simultaneous nuisance and correlated feature removal.

Findings

Outperforms existing methods in clustering tasks.

Effectively removes nuisance and correlated features.

Achieves state-of-the-art clustering results.

Abstract

Modern datasets often contain large subsets of correlated features and nuisance features, which are not or loosely related to the main underlying structures of the data. Nuisance features can be identified using the Laplacian score criterion, which evaluates the importance of a given feature via its consistency with the Graph Laplacians' leading eigenvectors. We demonstrate that in the presence of large numbers of nuisance features, the Laplacian must be computed on the subset of selected features rather than on the complete feature set. To do this, we propose a fully differentiable approach for unsupervised feature selection, utilizing the Laplacian score criterion to avoid the selection of nuisance features. We employ an autoencoder architecture to cope with correlated features, trained to reconstruct the data from the subset of selected features. Building on the recently proposed concrete layer that allows controlling for the number of selected features via architectural design, simplifying the optimization process. Experimenting on several real-world datasets, we demonstrate that our proposed approach outperforms similar approaches designed to avoid only correlated or nuisance features, but not both. Several state-of-the-art clustering results are reported.