TEN: Table Explicitization, Neurosymbolically

TL;DR

TEN is a neurosymbolic method that combines large language models with symbolic verification to accurately extract and verify tabular data from semi-structured text, outperforming neural-only approaches.

Contribution

The paper introduces TEN, a novel neurosymbolic framework that integrates chain-of-thought prompting and symbolic checking for improved table extraction from semi-structured text.

Findings

TEN achieves higher exact match accuracy than neural baselines.

TEN significantly reduces hallucination rates in table extraction.

User study shows TEN's tables are rated more accurate and preferred for verification.

Abstract

We present a neurosymbolic approach, TEN, for extracting tabular data from semistructured input text. This task is particularly challenging for text input that does not use special delimiters consistently to separate columns and rows. Purely neural approaches perform poorly due to hallucinations and their inability to enforce hard constraints. TEN uses Structural Decomposition prompting - a specialized chain-of-thought prompting approach - on a large language model (LLM) to generate an initial table, and thereafter uses a symbolic checker to evaluate not only the well-formedness of that table, but also detect cases of hallucinations or forgetting. The output of the symbolic checker is processed by a critique-LLM to generate guidance for fixing the table, which is presented to the original LLM in a self-debug loop. Our extensive experiments demonstrate that TEN significantly outperforms purely neural baselines across multiple datasets and metrics, achieving significantly higher exact match accuracy and substantially reduced hallucination rates. A 21-participant user study further confirms that TEN's tables are rated significantly more accurate (mean score: 5.0 vs 4.3; p = 0.021), and are consistently preferred for ease of verification and correction, with participants favoring our method in over 60% of the cases.