Plugging Schema Graph into Multi-Table QA: A Human-Guided Framework for Reducing LLM Reliance

TL;DR

This paper introduces a human-guided, graph-based framework for multi-table question answering that explicitly encodes schema links to improve reasoning over complex, real-world tables, reducing reliance on unreliable LLM schema linking.

Contribution

It presents a novel human-curated relational graph approach for multi-table QA, enabling interpretable reasoning and better handling of complex industrial data.

Findings

Effective on standard benchmarks and large-scale industrial datasets.

Outperforms existing semantic similarity-based methods.

First multi-table QA system applied to complex real-world data.

Abstract

Large language models (LLMs) have shown promise in table Question Answering (Table QA). However, extending these capabilities to multi-table QA remains challenging due to unreliable schema linking across complex tables. Existing methods based on semantic similarity work well only on simplified hand-crafted datasets and struggle to handle complex, real-world scenarios with numerous and diverse columns. To address this, we propose a graph-based framework that leverages human-curated relational knowledge to explicitly encode schema links and join paths. Given a natural language query, our method searches on graph to construct interpretable reasoning chains, aided by pruning and sub-path merging strategies to enhance efficiency and coherence. Experiments on both standard benchmarks and a realistic, large-scale dataset demonstrate the effectiveness of our approach. To our knowledge, this is the first multi-table QA system applied to truly complex industrial tabular data.