Robust Multi-class Feature Selection via $l_{2,0}$-Norm Regularization Minimization

TL;DR

This paper introduces a novel homotopy iterative hard threshold method for direct multi-class feature selection using $l_{2,0}$-norm regularization, achieving exact sparsity and high accuracy with fewer features.

Contribution

It proposes a new direct solution for $l_{2,0}$-norm regularization in multi-class feature selection, including an accelerated version for improved efficiency.

Findings

Achieves higher classification accuracy than existing methods.

Uses fewer features while maintaining performance.

Demonstrates robustness to parameter variations.

Abstract

Feature selection is an important data pre-processing in data mining and machine learning, which can reduce feature size without deteriorating model's performance. Recently, sparse regression based feature selection methods have received considerable attention due to their good performance. However, because the $l_{2,0}$-norm regularization term is non-convex, this problem is very hard to solve. In this paper, unlike most of the other methods which only solve the approximate problem, a novel method based on homotopy iterative hard threshold (HIHT) is proposed to solve the $l_{2,0}$-norm regularization least square problem directly for multi-class feature selection, which can produce exact row-sparsity solution for the weights matrix. What'more, in order to reduce the computational time of HIHT, an acceleration version of HIHT (AHIHT) is derived. Extensive experiments on eight biological datasets show that the proposed method can achieve higher classification accuracy (ACC) with fewest number of selected features (No.fea) comparing with the approximate convex counterparts and state-of-the-art feature selection methods. The robustness of classification accuracy to the regularization parameter and the number of selected feature are also exhibited.