Seek and Solve Reasoning for Table Question Answering

TL;DR

This paper introduces a Seek-and-Solve pipeline that enhances table question answering by leveraging LLMs' reasoning, focusing on the reasoning process itself rather than task simplification, leading to improved performance.

Contribution

The paper proposes a novel Seek-and-Solve approach with SS-CoT that integrates reasoning stages and distills a single-step prompt for better TQA performance.

Findings

Improved accuracy in table question answering tasks.

Enhanced reasoning capabilities of LLMs through the proposed pipeline.

Efficient approach requiring fewer task simplifications.

Abstract

The complexities of table structures and question logic make table-based question answering (TQA) tasks challenging for Large Language Models (LLMs), often requiring task simplification before solving. This paper reveals that the reasoning process during task simplification may be more valuable than the simplified tasks themselves and aims to improve TQA performance by leveraging LLMs' reasoning capabilities. We propose a Seek-and-Solve pipeline that instructs the LLM to first seek relevant information and then answer questions, integrating these two stages at the reasoning level into a coherent Seek-and-Solve Chain of Thought (SS-CoT). Additionally, we distill a single-step TQA-solving prompt from this pipeline, using demonstrations with SS-CoT paths to guide the LLM in solving complex TQA tasks under In-Context Learning settings. Our experiments show that our approaches result in improved performance and reliability while being efficient. Our findings emphasize the importance of eliciting LLMs' reasoning capabilities to handle complex TQA tasks effectively.