Linear Models, Variable Selection, Artificial Intelligence

TL;DR

This paper introduces an AI-based method using an ANN trained on OLS estimates for variable selection in linear regression, demonstrating competitive accuracy through simulations and real data application.

Contribution

It presents a novel AI approach to variable selection in linear models, leveraging an ANN trained on OLS estimates, and compares its performance with traditional methods.

Findings

The AI approach achieves high accuracy across various sample sizes and variances.

Simulation results show competitive performance against traditional methods like LASSO and BIC.

The pretrained ANN and datasets are available on GitHub for practical use.

Abstract

Variable selection in linear regression models has been a problem since hypothesis testing began. Which variables to include or exclude from a model is not an easy task. Techniques such as Forward, Back ward, Stepwise Regression sequentially add or delete variables from a model. Penalized likelihood methods such as AIC, BIC, etc. seek to choose variables that have a significant contribution to the likelihood. Penalized sum of square methods such as LASSO and Elastic Net have been used to penalize small coefficients to only allow variables with large coefficients in the model. This work introduces an Artificial Intelligence approach to model selection where an ANN is trained to determine the significance of the variables based on OLS estimates. A simulation study shows the accuracy across various sample sizes and variances. Furthermore, a simulation study is conducted to compare the performance of the approach against Forward, Backward, AIC, BIC and LASSO. The approach is illustrated using a dataset from the World Health Organization regarding Life Expectancy. A github link is provided to the pretrained ANN that can handle up to 100 predictor variables, the original WHO dataset and the subset used in this work.