DIWIFT: Discovering Instance-wise Influential Features for Tabular Data

TL;DR

DIWIFT introduces a novel influence function-based method for instance-wise feature selection in tabular data, improving flexibility and robustness over traditional global methods by accounting for scenario-specific influential features.

Contribution

The paper proposes DIWIFT, a new influence function-based approach utilizing self-attention networks for instance-wise feature selection in tabular data, addressing variability across scenarios.

Findings

DIWIFT outperforms existing methods on synthetic datasets.

DIWIFT demonstrates robustness and flexibility in real-world applications.

Experimental results validate the effectiveness of DIWIFT.

Abstract

Tabular data is one of the most common data storage formats behind many real-world web applications such as retail, banking, and e-commerce. The success of these web applications largely depends on the ability of the employed machine learning model to accurately distinguish influential features from all the predetermined features in tabular data. Intuitively, in practical business scenarios, different instances should correspond to different sets of influential features, and the set of influential features of the same instance may vary in different scenarios. However, most existing methods focus on global feature selection assuming that all instances have the same set of influential features, and few methods considering instance-wise feature selection ignore the variability of influential features in different scenarios. In this paper, we first introduce a new perspective based on the influence function for instance-wise feature selection, and give some corresponding theoretical insights, the core of which is to use the influence function as an indicator to measure the importance of an instance-wise feature. We then propose a new solution for discovering instance-wise influential features in tabular data (DIWIFT), where a self-attention network is used as a feature selection model and the value of the corresponding influence function is used as an optimization objective to guide the model. Benefiting from the advantage of the influence function, i.e., its computation does not depend on a specific architecture and can also take into account the data distribution in different scenarios, our DIWIFT has better flexibility and robustness. Finally, we conduct extensive experiments on both synthetic and real-world datasets to validate the effectiveness of our DIWIFT.