Bi-Sparse Unsupervised Feature Selection

TL;DR

This paper introduces BSUFS, a bi-sparse PCA-based method for unsupervised feature selection that effectively filters noise and selects relevant features using combined $,p$-norm and $,q$-norm regularizations.

Contribution

The paper proposes a novel bi-sparse PCA framework with an efficient optimization algorithm, extending existing methods and demonstrating improved feature selection performance.

Findings

BSUFS outperforms existing methods on synthetic and real datasets.

Bi-sparse optimization enhances feature relevance and noise filtering.

The proposed algorithm is computationally efficient and effective.

Abstract

To deal with high-dimensional unlabeled datasets in many areas, principal component analysis (PCA) has become a rising technique for unsupervised feature selection (UFS). However, most existing PCA-based methods only consider the structure of datasets by embedding a single sparse regularization or constraint on the transformation matrix. In this paper, we introduce a novel bi-sparse method called BSUFS to improve the performance of UFS. The core idea of BSUFS is to incorporate $\ell_{2,p}$-norm and $\ell_q$-norm into the classical PCA, which enables our method to select relevant features and filter out irrelevant noises, thereby obtaining discriminative features. Here, the parameters $p$ and $q$ are within the range of $[0, 1)$. Therefore, BSUFS not only constructs a unified framework for bi-sparse optimization, but also includes some existing works as special cases. To solve the resulting non-convex model, we propose an efficient proximal alternating minimization (PAM) algorithm using Stiefel manifold optimization and sparse optimization techniques. In addition, the computational complexity analysis is presented. Extensive numerical experiments on synthetic and real-world datasets demonstrate the effectiveness of our proposed BSUFS. The results reveal the advantages of bi-sparse optimization in feature selection and show its potential for other fields in image processing. Our code is available at https://github.com/xianchaoxiu/BSUFS.