A Transparent and Nonlinear Method for Variable Selection

TL;DR

This paper introduces a transparent, nonlinear variable selection method that groups predictors into meaningful subsets, improving interpretability and prediction accuracy in high-dimensional data.

Contribution

It proposes a novel three-step heuristic for decoupling predictor information and a nonlinear partial correlation for better dependence detection, enhancing interpretability and performance.

Findings

Outperforms state-of-the-art methods in prediction accuracy

Provides more interpretable variable selection results

Effective in high-dimensional, nonlinear settings

Abstract

Variable selection is a procedure to attain the truly important predictors from inputs. Complex nonlinear dependencies and strong coupling pose great challenges for variable selection in high-dimensional data. In addition, real-world applications have increased demands for interpretability of the selection process. A pragmatic approach should not only attain the most predictive covariates, but also provide ample and easy-to-understand grounds for removing certain covariates. In view of these requirements, this paper puts forward an approach for transparent and nonlinear variable selection. In order to transparently decouple information within the input predictors, a three-step heuristic search is designed, via which the input predictors are grouped into four subsets: the relevant to be selected, and the uninformative, redundant, and conditionally independent to be removed. A nonlinear partial correlation coefficient is introduced to better identify the predictors which have nonlinear functional dependence with the response. The proposed method is model-free and the selected subset can be competent input for commonly used predictive models. Experiments demonstrate the superior performance of the proposed method against the state-of-the-art baselines in terms of prediction accuracy and model interpretability.