ABM: an automatic supervised feature engineering method for loss based models based on group and fused lasso

TL;DR

ABM is an innovative supervised feature engineering method that automatically selects optimal binning points for variables, integrating feature engineering, variable selection, and model training, applicable to various loss-based models including neural networks.

Contribution

The paper introduces ABM, a novel end-to-end supervised binning method based on group and fused lasso, enhancing feature engineering and variable selection simultaneously.

Findings

Effective automatic binning improves model performance.

Applicable to diverse loss-based models including deep neural networks.

Open-source implementation available in R and Python.

Abstract

A vital problem in solving classification or regression problem is to apply feature engineering and variable selection on data before fed into models.One of a most popular feature engineering method is to discretisize continous variable with some cutting points,which is refered to as bining processing.Good cutting points are important for improving model's ability, because wonderful bining may ignore some noisy variance in continous variable range and keep useful leveled information with good ordered encodings.However, to our best knowledge a majority of cutting point selection is done via researchers domain knownledge or some naive methods like equal-width cutting or equal-frequency cutting.In this paper we propose an end-to-end supervised cutting point selection method based on group and fused lasso along with the automatically variable selection effect.We name our method \textbf{ABM}(automatic bining machine). We firstly cut each variable range into fine grid bins and train model with our group and group fused lasso regularization on each successive bins.It is a method that integrates feature engineering,variable selection and model training simultanously.And one more inspiring thing is that the method is flexible such that it can be taken into a bunch of loss function based model including deep neural networks.We have also implemented the method in R and open the source code to other researchers.A Python version will also meet the community in days.