Interpretable LLM-based Table Question Answering

TL;DR

This paper introduces POS, a novel method for interpretable table question answering using SQL decomposition, which enhances explanation quality, efficiency, and robustness while maintaining competitive accuracy.

Contribution

POS is the first approach to decompose questions into executable SQL steps for transparent decision-making in Table QA, reducing LLM calls and improving interpretability.

Findings

POS produces the highest-quality explanations among compared methods.

POS achieves competitive accuracy on standard benchmarks.

POS requires up to 25x fewer LLM calls and table queries.

Abstract

Interpretability in Table Question Answering (Table QA) is critical, especially in high-stakes domains like finance and healthcare. While recent Table QA approaches based on Large Language Models (LLMs) achieve high accuracy, they often produce ambiguous explanations of how answers are derived. We propose Plan-of-SQLs (POS), a new Table QA method that makes the model's decision-making process interpretable. POS decomposes a question into a sequence of atomic steps, each directly translated into an executable SQL command on the table, thereby ensuring that every intermediate result is transparent. Through extensive experiments, we show that: First, POS generates the highest-quality explanations among compared methods, which markedly improves the users' ability to simulate and verify the model's decisions. Second, when evaluated on standard Table QA benchmarks (TabFact, WikiTQ, and FeTaQA), POS achieves QA accuracy that is competitive to existing methods, while also offering greater efficiency-requiring significantly fewer LLM calls and table database queries (up to 25x fewer)-and more robust performance on large-sized tables. Finally, we observe high agreement (up to 90.59% in forward simulation) between LLMs and human users when making decisions based on the same explanations, suggesting that LLMs could serve as an effective proxy for humans in evaluating Table QA explanations.