Towards Benchmarking Foundation Models for Tabular Data With Text

TL;DR

This paper evaluates how foundation models for tabular data can incorporate textual features, proposing strategies and benchmarking their performance on real-world datasets with meaningful text, advancing the evaluation of multimodal tabular models.

Contribution

It introduces ablation strategies for adding text to tabular models and benchmarks state-of-the-art models on datasets with rich textual features.

Findings

Textual features improve model performance in certain tasks.

Benchmarking reveals strengths and limitations of current models with text.

Strategies for integrating text are effective and simple.

Abstract

Foundation models for tabular data are rapidly evolving, with increasing interest in extending them to support additional modalities such as free-text features. However, existing benchmarks for tabular data rarely include textual columns, and identifying real-world tabular datasets with semantically rich text features is non-trivial. We propose a series of simple yet effective ablation-style strategies for incorporating text into conventional tabular pipelines. Moreover, we benchmark how state-of-the-art tabular foundation models can handle textual data by manually curating a collection of real-world tabular datasets with meaningful textual features. Our study is an important step towards improving benchmarking of foundation models for tabular data with text.