ReasonTabQA: A Comprehensive Benchmark for Table Question Answering from Real World Industrial Scenarios

TL;DR

ReasonTabQA introduces a large-scale bilingual benchmark for industrial table question answering, emphasizing complex multi-table reasoning and explicit reasoning chains, to challenge and improve LLM capabilities in real-world scenarios.

Contribution

The paper presents ReasonTabQA, a comprehensive industrial TableQA benchmark with detailed annotations and reasoning chains, and proposes TabCodeRL, a reinforcement learning approach for logical reasoning.

Findings

TabCodeRL improves performance on open-source LLMs.

Persistent performance gap highlights industrial TableQA complexity.

ReasonTabQA covers 30 industry domains with 1,932 tables.

Abstract

Recent advancements in Large Language Models (LLMs) have significantly catalyzed table-based question answering (TableQA). However, existing TableQA benchmarks often overlook the intricacies of industrial scenarios, which are characterized by multi-table structures, nested headers, and massive scales. These environments demand robust table reasoning through deep structured inference, presenting a significant challenge that remains inadequately addressed by current methodologies. To bridge this gap, we present ReasonTabQA, a large-scale bilingual benchmark encompassing 1,932 tables across 30 industry domains such as energy and automotive. ReasonTabQA provides high-quality annotations for both final answers and explicit reasoning chains, supporting both thinking and no-thinking paradigms. Furthermore, we introduce TabCodeRL, a reinforcement learning method that leverages table-aware verifiable rewards to guide the generation of logical reasoning paths. Extensive experiments on ReasonTabQA and 4 TableQA datasets demonstrate that while TabCodeRL yields substantial performance gains on open-source LLMs, the persistent performance gap on ReasonTabQA underscores the inherent complexity of real-world industrial TableQA.