Toward Efficient Automated Feature Engineering

TL;DR

This paper introduces a reinforcement learning-based framework for automated feature engineering that significantly improves efficiency and maintains high effectiveness, enabling faster deployment on large-scale datasets.

Contribution

The work proposes a novel AFE framework with a feature pre-evaluation model and a two-stage policy training strategy to enhance efficiency without sacrificing performance.

Findings

Achieved 2.9% higher average performance

Doubled the computational efficiency compared to existing methods

Validated on 36 datasets for classification and regression

Abstract

Automated Feature Engineering (AFE) refers to automatically generate and select optimal feature sets for downstream tasks, which has achieved great success in real-world applications. Current AFE methods mainly focus on improving the effectiveness of the produced features, but ignoring the low-efficiency issue for large-scale deployment. Therefore, in this work, we propose a generic framework to improve the efficiency of AFE. Specifically, we construct the AFE pipeline based on reinforcement learning setting, where each feature is assigned an agent to perform feature transformation \com{and} selection, and the evaluation score of the produced features in downstream tasks serve as the reward to update the policy. We improve the efficiency of AFE in two perspectives. On the one hand, we develop a Feature Pre-Evaluation (FPE) Model to reduce the sample size and feature size that are two main factors on undermining the efficiency of feature evaluation. On the other hand, we devise a two-stage policy training strategy by running FPE on the pre-evaluation task as the initialization of the policy to avoid training policy from scratch. We conduct comprehensive experiments on 36 datasets in terms of both classification and regression tasks. The results show $2.9\%$ higher performance in average and 2x higher computational efficiency comparing to state-of-the-art AFE methods.