ODUTQA-MDC: A Task for Open-Domain Underspecified Tabular QA with Multi-turn Dialogue-based Clarification

TL;DR

This paper introduces ODUTQA-MDC, a new benchmark and framework for open-domain, underspecified tabular question answering that uses multi-turn dialogue for clarification.

Contribution

It presents the first comprehensive benchmark and a multi-agent framework for interactive, clarification-based tabular QA with large-scale data.

Findings

The benchmark includes 209 tables and 25,105 QA pairs.

The MAIC-TQA framework effectively detects ambiguities and refines answers through dialogue.

Experiments demonstrate the benchmark and framework's effectiveness in advancing conversational tabular QA.

Abstract

The advancement of large language models (LLMs) has enhanced tabular question answering (Tabular QA), yet they struggle with open-domain queries exhibiting underspecified or uncertain expressions. To address this, we introduce the ODUTQA-MDC task and the first comprehensive benchmark to tackle it. This benchmark includes: (1) a large-scale ODUTQA dataset with 209 tables and 25,105 QA pairs; (2) a fine-grained labeling scheme for detailed evaluation; and (3) a dynamic clarification interface that simulates user feedback for interactive assessment. We also propose MAIC-TQA, a multi-agent framework that excels at detecting ambiguities, clarifying them through dialogue, and refining answers. Experiments validate our benchmark and framework, establishing them as a key resource for advancing conversational, underspecification-aware Tabular QA research.