Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework

TL;DR

This paper introduces a self-training framework for unsupervised complex table reasoning that generates synthetic data with complex logic, significantly reducing the need for manual annotations and improving performance on real-world tasks.

Contribution

The paper presents a novel self-training framework, UCTR-ST, that leverages synthetic data generation and multiple modules to enhance unsupervised reasoning over tables.

Findings

Achieves over 90% of supervised model performance on various tasks.

Reduces dependence on manual annotation for complex table reasoning.

Enhances supervised models' performance in low-resource domains.

Abstract

Structured tabular data is a fundamental data type in numerous fields, and the capacity to reason over tables is crucial for answering questions and validating hypotheses. However, constructing labeled data for complex reasoning tasks is labor intensive, and the quantity of annotated data remains insufficient to support the intricate demands of real-world applications. To address the insufficient annotation challenge, we present a self-training framework for unsupervised complex tabular reasoning (UCTR-ST) by generating diverse synthetic data with complex logic. Specifically, UCTR-ST incorporates several essential techniques: we aggregate diverse programs and execute them on tables based on a "Program-Management" component, and we bridge the gap between programs and text with a powerful "Program-Transformation" module that generates natural language sentences with complex logic. Furthermore, we optimize the procedure using a "Table-Text Manipulator" to handle joint table-text reasoning scenarios. The entire framework utilizes self-training techniques to leverage the unlabeled training data, which results in significant performance improvements when tested on real-world data. Experimental results demonstrate that UCTRST achieves above 90% of the supervised model performance on different tasks and domains, reducing the dependence on manual annotation. Additionally, our approach can serve as a data augmentation technique, significantly boosting the performance of supervised models in low-resourced domains.