FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data

TL;DR

FIVES is a novel method that efficiently generates high-order interactive features for tabular data by searching feature graphs with a GNN, improving performance and interpretability in real-world applications.

Contribution

FIVES introduces a graph neural network-based search strategy for interactive feature generation, balancing interpretability and efficiency in large-scale tabular data.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Enhances recommender system performance in Taobao.

Validated through online A/B testing and deployment.

Abstract

High-order interactive features capture the correlation between different columns and thus are promising to enhance various learning tasks on ubiquitous tabular data. To automate the generation of interactive features, existing works either explicitly traverse the feature space or implicitly express the interactions via intermediate activations of some designed models. These two kinds of methods show that there is essentially a trade-off between feature interpretability and search efficiency. To possess both of their merits, we propose a novel method named Feature Interaction Via Edge Search (FIVES), which formulates the task of interactive feature generation as searching for edges on the defined feature graph. Specifically, we first present our theoretical evidence that motivates us to search for useful interactive features with increasing order. Then we instantiate this search strategy by optimizing both a dedicated graph neural network (GNN) and the adjacency tensor associated with the defined feature graph. In this way, the proposed FIVES method simplifies the time-consuming traversal as a typical training course of GNN and enables explicit feature generation according to the learned adjacency tensor. Experimental results on both benchmark and real-world datasets show the advantages of FIVES over several state-of-the-art methods. Moreover, the interactive features identified by FIVES are deployed on the recommender system of Taobao, a worldwide leading e-commerce platform. Results of an online A/B testing further verify the effectiveness of the proposed method FIVES, and we further provide FIVES as AI utilities for the customers of Alibaba Cloud.