ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context

TL;DR

ProTrix introduces a planning-based framework for reasoning over tables with sentence context, improving accuracy, generalization, and interpretability in tabular question answering tasks.

Contribution

The paper presents a novel Plan-then-Reason framework and the ProTrix model family, enhancing table reasoning through planning, instruction tuning, and efficient fine-tuning methods.

Findings

GPT-3.5-Turbo with ProTrix surpasses baselines without self-consistency.

ProTrix generalizes well with only 6k training instances.

ProTrix provides accurate, faithful explanations for complex questions.

Abstract

Tables play a crucial role in conveying information in various domains. We propose a Plan-then-Reason framework to answer different types of user queries over tables with sentence context. The framework first plans the reasoning paths over the context, then assigns each step to program-based or textual reasoning to reach the final answer. This framework enhances the table reasoning abilities for both in-context learning and fine-tuning methods. GPT-3.5-Turbo following Plan-then-Reason framework surpasses other prompting baselines without self-consistency while using less API calls and in-context demonstrations. We also construct an instruction tuning set TrixInstruct to evaluate the effectiveness of fine-tuning with this framework. We present ProTrix model family by finetuning models on TrixInstruct. Our experiments show that ProTrix family generalizes to diverse unseen tabular tasks with only 6k training instances. We further demonstrate that ProTrix can generate accurate and faithful explanations to answer complex free-form questions. Our work underscores the importance of the planning and reasoning abilities towards a model over tabular tasks with generalizability and interpretability. We open-source our dataset and models at https://github.com/WilliamZR/ProTrix.