No Universal Prompt: Unifying Reasoning through Adaptive Prompting for Temporal Table Reasoning

TL;DR

This paper investigates how different prompting methods affect large language models' reasoning over temporal tables, revealing that no single method is best and proposing an adaptive framework, SEAR, that improves reasoning across diverse table types.

Contribution

The paper introduces SEAR, an adaptive prompting framework that dynamically adjusts to context, significantly enhancing reasoning performance over various temporal table structures.

Findings

Performance varies with table and context structure.

SEAR outperforms baseline prompting methods.

Unified table representation improves reasoning.

Abstract

Temporal Table Reasoning is a critical challenge for Large Language Models (LLMs), requiring effective reasoning to extract relevant insights. Despite existence of multiple prompting methods, their impact on table reasoning remains largely unexplored. Furthermore, model performance varies drastically across different table and context structures, making it difficult to determine an optimal approach. This work investigates multiple prompting technique on diverse table types to determine that performance depends on factors such as entity type, table structure, requirement of additional context and question complexity, with "NO" single method consistently outperforming others. To address this, we introduce SEAR, an adaptive prompting framework inspired by human reasoning that dynamically adjusts to context and integrates structured reasoning. Our results demonstrate that SEAR achieves superior performance across all table types compared to baseline prompting techniques. Additionally, we explore the impact of table structure refactoring, finding that a unified representation enhances model reasoning.