Table Retrieval May Not Necessitate Table-specific Model Design

TL;DR

This paper investigates whether specialized table models are necessary for table retrieval tasks, finding that simple text-based models perform comparably or better than complex, table-specific models, with structure often playing a negligible role.

Contribution

The study demonstrates that generic text-based retrieval models can effectively handle table retrieval, challenging the need for complex, table-specific model designs.

Findings

Text-based DPR performs well without table-specific design.

Table structure plays a negligible role in over 70% of cases.

Adding explicit table structure encoding does not significantly improve performance.

Abstract

Tables are an important form of structured data for both human and machine readers alike, providing answers to questions that cannot, or cannot easily, be found in texts. Recent work has designed special models and training paradigms for table-related tasks such as table-based question answering and table retrieval. Though effective, they add complexity in both modeling and data acquisition compared to generic text solutions and obscure which elements are truly beneficial. In this work, we focus on the task of table retrieval, and ask: "is table-specific model design necessary for table retrieval, or can a simpler text-based model be effectively used to achieve a similar result?" First, we perform an analysis on a table-based portion of the Natural Questions dataset (NQ-table), and find that structure plays a negligible role in more than 70% of the cases. Based on this, we experiment with a general Dense Passage Retriever (DPR) based on text and a specialized Dense Table Retriever (DTR) that uses table-specific model designs. We find that DPR performs well without any table-specific design and training, and even achieves superior results compared to DTR when fine-tuned on properly linearized tables. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. However, none of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.