Utilizing Training Data to Improve LLM Reasoning for Tabular Understanding

TL;DR

This paper introduces LRTab, a novel prompting-based method that combines training data insights with retrieval techniques to enhance large language model reasoning for tabular data understanding, outperforming previous methods.

Contribution

LRTab integrates training data insights with retrieval to improve LLM reasoning in tabular tasks, balancing generalizability and dataset-specific learning.

Findings

LRTab outperforms previous baselines on WikiTQ and Tabfact datasets.

LRTab is interpretable and cost-efficient.

Retrieving relevant prompt conditions enhances reasoning accuracy.

Abstract

Automated tabular understanding and reasoning are essential tasks for data scientists. Recently, Large language models (LLMs) have become increasingly prevalent in tabular reasoning tasks. Previous work focuses on (1) finetuning LLMs using labeled data or (2) Training-free prompting LLM agents using chain-of-thought (CoT). Finetuning offers dataset-specific learning at the cost of generalizability. Training-free prompting is highly generalizable but does not take full advantage of training data. In this paper, we propose a novel prompting-based reasoning approach, Learn then Retrieve: LRTab, which integrates the benefits of both by retrieving relevant information learned from training data. We first use prompting to obtain CoT responses over the training data. For incorrect CoTs, we prompt the LLM to predict Prompt Conditions to avoid the error, learning insights from the data. We validate the effectiveness of Prompt Conditions using validation data. Finally, at inference time, we retrieve the most relevant Prompt Conditions for additional context for table understanding. We provide comprehensive experiments on WikiTQ and Tabfact, showing that LRTab is interpretable, cost-efficient, and can outperform previous baselines in tabular reasoning.