Fine-Grained Table Retrieval Through the Lens of Complex Queries

TL;DR

This paper introduces DCTR, a fine-grained table retrieval method that decomposes complex natural language queries and considers global database connectivity, improving retrieval for open-domain question answering over relational databases.

Contribution

The paper proposes DCTR, a novel retrieval approach that enhances handling of complex, composite queries and densely connected databases in open-domain QA systems.

Findings

DCTR improves retrieval accuracy for complex queries.

DCTR is robust with highly composite queries.

DCTR performs well on industry-aligned benchmarks.

Abstract

Enabling question answering over tables and databases in natural language has become a key capability in the democratization of insights from tabular data sources. These systems first require retrieval of data that is relevant to a given natural language query, for which several methods have been introduced. In this work we present and study a table retrieval mechanism devising fine-grained typed query decomposition and global connectivity-awareness (DCTR), to handle the challenges induced by open-domain question answering over relational databases in complex usage contexts. We evaluate the effectiveness of the two mechanisms through the lens of retrieval complexity which we measure along the axes of query- and data complexity. Our analyses over industry-aligned benchmarks illustrate the robustness of DCTR for highly composite queries and densely connected databases.