TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments
Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Yu-Feng Li, Lan-Zhe Guo

TL;DR
This paper introduces TabFSBench, a comprehensive benchmark for evaluating the impact of feature shifts in open environments on tabular data models, highlighting their limited robustness and the linear relationship between feature importance and performance degradation.
Contribution
It presents the first systematic study and benchmark for feature shifts in tabular data, including evaluation of large language models and insights into model robustness.
Findings
Most tabular models have limited applicability under feature shifts.
Feature importance shifts linearly relate to performance degradation.
Model performance in closed environments correlates with performance under feature shifts.
Abstract
Tabular data is widely utilized in various machine learning tasks. Current tabular learning research predominantly focuses on closed environments, while in real-world applications, open environments are often encountered, where distribution and feature shifts occur, leading to significant degradation in model performance. Previous research has primarily concentrated on mitigating distribution shifts, whereas feature shifts, a distinctive and unexplored challenge of tabular data, have garnered limited attention. To this end, this paper conducts the first comprehensive study on feature shifts in tabular data and introduces the first tabular feature-shift benchmark (TabFSBench). TabFSBench evaluates impacts of four distinct feature-shift scenarios on four tabular model categories across various datasets and assesses the performance of large language models (LLMs) and tabular LLMs in the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Speech Recognition and Synthesis
MethodsSparse Evolutionary Training
