ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples

TL;DR

ReasTAP is a pre-training method that injects seven high-level table reasoning skills into models using synthetic question-answer pairs, significantly improving performance across multiple table-related tasks.

Contribution

The paper introduces a novel pre-training approach that incorporates multiple reasoning skills through synthetic data, avoiding complex architecture modifications.

Findings

Achieves state-of-the-art results on four table reasoning benchmarks.

Significantly improves performance in low-resource settings.

Effectively models diverse reasoning skills without complex architectures.

Abstract

Reasoning over tabular data requires both table structure understanding and a broad set of table reasoning skills. Current models with table-specific architectures and pre-training methods perform well on understanding table structures, but they still struggle with tasks that require various table reasoning skills. In this work, we develop ReasTAP to show that high-level table reasoning skills can be injected into models during pre-training without a complex table-specific architecture design. We define 7 table reasoning skills, such as numerical operation, temporal comparison, and conjunction. Each reasoning skill is associated with one example generator, which synthesizes questions over semi-structured tables according to the sampled templates. We model the table pre-training task as a sequence generation task and pre-train ReasTAP to generate precise answers to the synthetic examples. ReasTAP is evaluated on four benchmarks covering three downstream tasks including: 1) WikiSQL and WTQ for Table Question Answering; 2) TabFact for Table Fact Verification; and 3) LogicNLG for Faithful Table-to-Text Generation. Experimental results demonstrate that ReasTAP achieves new state-of-the-art performance on all benchmarks and delivers a significant improvement on low-resource setting. Our code is publicly available at https://github.com/Yale-LILY/ReasTAP.