A Combined Approach To Detect Key Variables In Thick Data Analytics

TL;DR

This paper proposes a permutation test-based method for selecting key predictor variables in machine learning, demonstrated through chemical analysis, and compares it with the Lasso technique.

Contribution

It introduces a novel permutation test approach for variable selection and evaluates its effectiveness against Lasso in industrial and chemical analysis contexts.

Findings

Permutation test approach effectively identifies significant variables.

The method shows competitive performance compared to Lasso.

Application to chemical analysis demonstrates practical utility.

Abstract

In machine learning one of the strategic tasks is the selection of only significant variables as predictors for the response(s). In this paper an approach is proposed which consists in the application of permutation tests on the candidate predictor variables in the aim of identifying only the most informative ones. Several industrial problems may benefit from such an approach, and an application in the field of chemical analysis is presented. A comparison is carried out between the approach proposed and Lasso, that is one of the most common alternatives for feature selection available in the literature.