Leveraging Data Recasting to Enhance Tabular Reasoning

TL;DR

This paper introduces a semi-automatic framework for recasting existing tabular datasets to generate challenging inference data, combining the diversity of human annotation with the scalability of synthetic methods, to improve reasoning models.

Contribution

The authors propose a novel recasting framework that transforms various existing datasets into tabular NLI data, enhancing data diversity and utility for training and evaluation.

Findings

Recast datasets improve model performance on tabular NLI tasks.

Recast data serve as effective evaluation benchmarks.

Models trained on recasted data perform well in zero-shot scenarios.

Abstract

Creating challenging tabular inference data is essential for learning complex reasoning. Prior work has mostly relied on two data generation strategies. The first is human annotation, which yields linguistically diverse data but is difficult to scale. The second category for creation is synthetic generation, which is scalable and cost effective but lacks inventiveness. In this research, we present a framework for semi-automatically recasting existing tabular data to make use of the benefits of both approaches. We utilize our framework to build tabular NLI instances from five datasets that were initially intended for tasks like table2text creation, tabular Q/A, and semantic parsing. We demonstrate that recasted data could be used as evaluation benchmarks as well as augmentation data to enhance performance on tabular NLI tasks. Furthermore, we investigate the effectiveness of models trained on recasted data in the zero-shot scenario, and analyse trends in performance across different recasted datasets types.