Bi-Level Unsupervised Feature Selection

TL;DR

This paper introduces BLUFS, a novel bi-level unsupervised feature selection method that combines spectral clustering and a new $ ext{l}_{2,0}$-norm constraint, improving feature selection and data structure preservation.

Contribution

It is the first to integrate a bi-level framework with the $ ext{l}_{2,0}$-norm for unsupervised feature selection, along with an efficient optimization algorithm and convergence analysis.

Findings

BLUFS outperforms existing methods in clustering accuracy.

BLUFS enhances feature selection effectiveness.

Experiments validate BLUFS's superiority on multiple datasets.

Abstract

Unsupervised feature selection (UFS) is an important task in data engineering. However, most UFS methods construct models from a single perspective and often fail to simultaneously evaluate feature importance and preserve their inherent data structure, thus limiting their performance. To address this challenge, we propose a novel bi-level unsupervised feature selection (BLUFS) method, including a clustering level and a feature level. Specifically, at the clustering level, spectral clustering is used to generate pseudo-labels for representing the data structure, while a continuous linear regression model is developed to learn the projection matrix. At the feature level, the $\ell_{2,0}$-norm constraint is imposed on the projection matrix for more effectively selecting features. To the best of our knowledge, this is the first work to combine a bi-level framework with the $\ell_{2,0}$-norm. To solve the proposed bi-level model, we design an efficient proximal alternating minimization (PAM) algorithm, whose subproblems either have explicit solutions or can be computed by fast solvers. Furthermore, we establish the convergence result and computational complexity. Finally, extensive experiments on two synthetic datasets and eight real datasets demonstrate the superiority of BLUFS in clustering and classification tasks.