LCEN: A Nonlinear, Interpretable Feature Selection and Machine Learning Algorithm

TL;DR

LCEN is a nonlinear, interpretable feature selection and machine learning algorithm that outperforms existing methods in accuracy, speed, and robustness across various datasets, including physical law discovery.

Contribution

Introduces LCEN, a novel nonlinear, interpretable method for feature selection and machine learning that is faster and more robust than existing approaches.

Findings

LCEN constructs sparse, accurate models across datasets.

LCEN is approximately 10 times faster than elastic net.

LCEN can rediscover physical laws from data.

Abstract

Interpretable models can have advantages over black-box models, and interpretability is essential for the application of machine learning in critical settings, such as aviation or medicine. This article introduces the LASSO-Clip-EN (LCEN) algorithm for nonlinear, interpretable feature selection and machine learning modeling. In a wide variety of artificial and empirical datasets, LCEN constructed sparse and frequently more accurate models than other methods, including sparse, nonlinear methods, on tested datasets. LCEN was empirically observed to be robust against many issues typically present in datasets and modeling, including noise, multicollinearity, and data scarcity. As a feature selection algorithm, LCEN matched or surpassed the thresholded elastic net but was, on average, 10.3-fold faster based on our experiments. LCEN for feature selection can also rediscover multiple physical laws from empirical data. As a machine learning algorithm, when tested on processes with no known physical laws, LCEN achieved better results than many other dense and sparse methods -- including being comparable to or better than ANNs on multiple datasets.