Tabular Embeddings for Tables with Bi-Dimensional Hierarchical Metadata and Nesting

TL;DR

This paper introduces specialized 2-D tabular embeddings that effectively encode complex hierarchical and nested metadata in tables, improving performance on structured data tasks.

Contribution

The paper presents a novel embedding method tailored for 2-D hierarchical and nested table metadata, explicitly modeling complex table structures.

Findings

Outperforms state-of-the-art models with MAP improvements up to 0.28

Achieves significant performance gains over GPT-4 LLM+RAG in MAP metric

Validated on 5 large-scale datasets across 3 downstream tasks

Abstract

Embeddings serve as condensed vector representations for real-world entities, finding applications in Natural Language Processing (NLP), Computer Vision, and Data Management across diverse downstream tasks. Here, we introduce novel specialized embeddings optimized, and explicitly tailored to encode the intricacies of complex 2-D context in tables, featuring horizontal, vertical hierarchical metadata, and nesting. To accomplish that we define the Bi-dimensional tabular coordinates, separate horizontal, vertical metadata and data contexts by introducing a new visibility matrix, encode units and nesting through the embeddings specifically optimized for mimicking intricacies of such complex structured data. Through evaluation on 5 large-scale structured datasets and 3 popular downstream tasks, we observed that our solution outperforms the state-of-the-art models with the significant MAP delta of up to 0.28. GPT-4 LLM+RAG slightly outperforms us with MRR delta of up to 0.1, while we outperform it with the MAP delta of up to 0.42.