$l_{2,p}$ Matrix Norm and Its Application in Feature Selection

TL;DR

This paper introduces a new mixed $l_{2,p}$ matrix norm for $0<p extless 1$, providing an efficient algorithm for its optimization and demonstrating improved feature selection in biological data over the traditional $l_{2,1}$ norm.

Contribution

It defines the $l_{2,p}$ matrix pseudo norm for $0<p extless 1$, extending sparse regularization to nonconvex cases and proposes a unified algorithm with proven convergence.

Findings

The $l_{2,p}$ norm enhances sparsity in feature selection.

Experimental results show improved feature selection with $0<p<1$.

The algorithm converges uniformly for all $p extless 1$.

Abstract

Recently, $l_{2,1}$ matrix norm has been widely applied to many areas such as computer vision, pattern recognition, biological study and etc. As an extension of $l_1$ vector norm, the mixed $l_{2,1}$ matrix norm is often used to find jointly sparse solutions. Moreover, an efficient iterative algorithm has been designed to solve $l_{2,1}$-norm involved minimizations. Actually, computational studies have showed that $l_p$-regularization ($0<p<1$) is sparser than $l_1$-regularization, but the extension to matrix norm has been seldom considered. This paper presents a definition of mixed $l_{2,p}$ $(p\in (0, 1])$ matrix pseudo norm which is thought as both generalizations of $l_p$ vector norm to matrix and $l_{2,1}$-norm to nonconvex cases $(0<p<1)$. Fortunately, an efficient unified algorithm is proposed to solve the induced $l_{2,p}$-norm $(p\in (0, 1])$ optimization problems. The convergence can also be uniformly demonstrated for all $p\in (0, 1]$. Typical $p\in (0,1]$ are applied to select features in computational biology and the experimental results show that some choices of $0<p<1$ do improve the sparse pattern of using $p=1$.