TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding

TL;DR

TabEmbed introduces a unified embedding model for tabular data, enabling classification and retrieval tasks by capturing structural and numerical features, and is evaluated on the new TabBench benchmark.

Contribution

The paper presents TabEmbed, the first generalist embedding model for tabular data that unifies classification and retrieval in a shared space, and introduces the TabBench benchmark.

Findings

TabEmbed outperforms existing text embedding models on TabBench.

Reformulating tabular tasks as semantic matching improves understanding.

Contrastive learning with hard negative mining enhances model performance.

Abstract

Foundation models have established unified representations for natural language processing, yet this paradigm remains largely unexplored for tabular data. Existing methods face fundamental limitations: LLM-based approaches lack retrieval-compatible vector outputs, whereas text embedding models often fail to capture tabular structure and numerical semantics. To bridge this gap, we first introduce the Tabular Embedding Benchmark (TabBench), a comprehensive suite designed to evaluate the tabular understanding capability of embedding models. We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space. By reformulating diverse tabular tasks as semantic matching problems, TabEmbed leverages large-scale contrastive learning with positive-aware hard negative mining to discern fine-grained structural and numerical nuances. Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models, establishing a new baseline for universal tabular representation learning. Code and datasets are publicly available at https://github.com/qiangminjie27/TabEmbed and https://huggingface.co/datasets/qiangminjie27/TabBench.