Tabular Embeddings for Tables with Bi-Dimensional Hierarchical Metadata and Nesting
Gyanendra Shrestha, Chutain Jiang, Sai Akula, Vivek Yannam, Anna, Pyayt, Michael Gubanov

TL;DR
This paper introduces specialized 2-D tabular embeddings that effectively encode complex hierarchical and nested metadata in tables, improving performance on structured data tasks.
Contribution
The paper presents a novel embedding method tailored for 2-D hierarchical and nested table metadata, explicitly modeling complex table structures.
Findings
Outperforms state-of-the-art models with MAP improvements up to 0.28
Achieves significant performance gains over GPT-4 LLM+RAG in MAP metric
Validated on 5 large-scale datasets across 3 downstream tasks
Abstract
Embeddings serve as condensed vector representations for real-world entities, finding applications in Natural Language Processing (NLP), Computer Vision, and Data Management across diverse downstream tasks. Here, we introduce novel specialized embeddings optimized, and explicitly tailored to encode the intricacies of complex 2-D context in tables, featuring horizontal, vertical hierarchical metadata, and nesting. To accomplish that we define the Bi-dimensional tabular coordinates, separate horizontal, vertical metadata and data contexts by introducing a new visibility matrix, encode units and nesting through the embeddings specifically optimized for mimicking intricacies of such complex structured data. Through evaluation on 5 large-scale structured datasets and 3 popular downstream tasks, we observed that our solution outperforms the state-of-the-art models with the significant MAP…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Database Systems and Queries · Data Quality and Management · Web Data Mining and Analysis
MethodsAttention Is All You Need · Absolute Position Encodings · Linear Layer · Layer Normalization · Byte Pair Encoding · Dense Connections · Residual Connection · Label Smoothing · Multi-Head Attention · Position-Wise Feed-Forward Layer
