ALTER: Augmentation for Large-Table-Based Reasoning

TL;DR

ALTER is a framework that enhances large-table reasoning in LLMs by augmenting data with minimal relevant information, improving scalability, robustness, and performance on reasoning benchmarks.

Contribution

It introduces a novel augmentation framework leveraging both natural language questions and semi-structured tables, addressing scalability issues in large-table reasoning with LLMs.

Findings

Outperforms existing methods on large-table reasoning benchmarks

Demonstrates robustness against data perturbations

Efficiently utilizes limited relevant data for augmentation

Abstract

While extensive research has explored the use of large language models (LLMs) for table-based reasoning, most approaches struggle with scalability when applied to large tables. To maintain the superior comprehension abilities of LLMs in these scenarios, we introduce ALTER(Augmentation for Large-Table-Based Reasoning)-a framework designed to harness the latent augmentation potential in both free-form natural language (NL) questions, via the query augmentor, and semi-structured tabular data, through the table augmentor. By utilizing only a small subset of relevant data from the table and supplementing it with pre-augmented schema, semantic, and literal information, ALTER achieves outstanding performance on table-based reasoning benchmarks. We also provide a detailed analysis of large-table scenarios, comparing different methods and various partitioning principles. In these scenarios, our method outperforms all other approaches and exhibits robustness and efficiency against perturbations.