Human-LLM Collaborative Feature Engineering for Tabular Data

TL;DR

This paper introduces a collaborative framework combining human expertise and large language models for more effective feature engineering in tabular data, improving performance and reducing cognitive load.

Contribution

It decouples operation proposal and selection, incorporating human feedback to better estimate utility and prioritize promising feature transformations.

Findings

Improved feature engineering performance on various datasets.

Reduced cognitive load for users during feature engineering.

Effective integration of human feedback enhances operation selection.

Abstract

Large language models (LLMs) are increasingly used to automate feature engineering in tabular learning. Given task-specific information, LLMs can propose diverse feature transformation operations to enhance downstream model performance. However, current approaches typically assign the LLM as a black-box optimizer, responsible for both proposing and selecting operations based solely on its internal heuristics, which often lack calibrated estimations of operation utility and consequently lead to repeated exploration of low-yield operations without a principled strategy for prioritizing promising directions. In this paper, we propose a human-LLM collaborative feature engineering framework for tabular learning. We begin by decoupling the transformation operation proposal and selection processes, where LLMs are used solely to generate operation candidates, while the selection is guided by explicitly modeling the utility and uncertainty of each proposed operation. Since accurate utility estimation can be difficult especially in the early rounds of feature engineering, we design a mechanism within the framework that selectively elicits and incorporates human expert preference feedback, comparing which operations are more promising, into the selection process to help identify more effective operations. Our evaluations on both the synthetic study and the real user study demonstrate that the proposed framework improves feature engineering performance across a variety of tabular datasets and reduces users' cognitive load during the feature engineering process.