Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning

TL;DR

Reasoning-Table introduces reinforcement learning to table reasoning tasks, achieving state-of-the-art results and improved robustness over traditional supervised fine-tuning methods across multiple benchmarks.

Contribution

This work is the first to apply reinforcement learning to table reasoning, outperforming supervised fine-tuning and larger models through tailored training and reward strategies.

Findings

Outperforms supervised fine-tuning on multiple benchmarks.

Achieves 68.3% on text-to-SQL BIRD dataset with a 7B model.

Enhances model robustness and generalization capabilities.

Abstract

Table reasoning, encompassing tasks such as table question answering, fact verification, and text-to-SQL, requires precise understanding of structured tabular data, coupled with numerical computation and code manipulation for effective inference. Supervised fine-tuning (SFT) approaches have achieved notable success but often struggle with generalization and robustness due to biases inherent in imitative learning. We introduce Reasoning-Table, the first application of reinforcement learning (RL) to table reasoning, achieving state-of-the-art performance. Through rigorous data preprocessing, reward design, and tailored training strategies, our method leverages simple rule-based outcome rewards to outperform SFT across multiple benchmarks. Unified training across diverse tasks enables Reasoning-Table to emerge as a robust table reasoning large language model, surpassing larger proprietary models like Claude-3.7-Sonnet by 4.0% on table reasoning benchmarks. The approach also achieves excellent performance on text-to-SQL tasks, reaching 68.3% performance on the BIRD dev dataset with a 7B model. Further experiments demonstrate that Reasoning-Table enhances the model's generalization capabilities and robustness.