TL;DR
TabEmb introduces a novel approach for table annotation by combining semantic embeddings from LLMs with structural relationship modeling, leading to improved performance over existing linearization-based methods.
Contribution
The paper presents TabEmb, a method that decouples semantic and structural encoding for tables, enhancing annotation accuracy and generalization.
Findings
TabEmb outperforms baseline models on multiple table annotation tasks.
Decoupling semantic and structural modeling improves representation quality.
Graph-based structural modeling enhances the semantic embeddings.
Abstract
Table annotation is crucial for making web and enterprise tables usable in downstream NLP applications. Unlike textual data where learning semantically rich token or sentence embeddings often suffice, tables are structured combinations of columns wherein useful representations must jointly capture column's semantics and the inter-column relationships. Existing models learn by linearizing the 2D table into a 1D token sequence and encoding it with pretrained language models (PLMs) such as BERT. However, this leads to limited semantic quality and weaker generalization to unseen or rare values compared to modern LLMs, and degraded structural modeling due to 2D-to-1D flattening and context-length constraints. We propose TabEmb, which directly targets these limitations by decoupling semantic encoding from structural modeling. An LLM first produces semantically rich embeddings for each column,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
