Regularized Multivariate Analysis Framework for Interpretable High-Dimensional Variable Selection

TL;DR

This paper introduces a regularized multivariate analysis framework that efficiently performs high-dimensional variable selection while maintaining feature uncorrelation, enhancing interpretability and outperforming existing methods.

Contribution

It proposes a novel approach using l-21 norm regularization to enable interpretable variable selection in multivariate analysis, preserving key properties of original methods.

Findings

The proposed method effectively selects variables in high-dimensional data.

It maintains uncorrelation among extracted features.

Experimental results show superior performance over state-of-the-art methods.

Abstract

Multivariate Analysis (MVA) comprises a family of well-known methods for feature extraction which exploit correlations among input variables representing the data. One important property that is enjoyed by most such methods is uncorrelation among the extracted features. Recently, regularized versions of MVA methods have appeared in the literature, mainly with the goal to gain interpretability of the solution. In these cases, the solutions can no longer be obtained in a closed manner, and more complex optimization methods that rely on the iteration of two steps are frequently used. This paper recurs to an alternative approach to solve efficiently this iterative problem. The main novelty of this approach lies in preserving several properties of the original methods, most notably the uncorrelation of the extracted features. Under this framework, we propose a novel method that takes advantage of the l-21 norm to perform variable selection during the feature extraction process. Experimental results over different problems corroborate the advantages of the proposed formulation in comparison to state of the art formulations.