TreeRare: Syntax Tree-Guided Retrieval and Reasoning for Knowledge-Intensive Question Answering

TL;DR

TreeRare introduces a syntax tree-guided framework for knowledge-intensive question answering, improving retrieval and reasoning accuracy by leveraging syntactic structures to guide localized evidence gathering and synthesis.

Contribution

It proposes a novel syntax tree-guided retrieval and reasoning framework that enhances complex question answering by addressing reasoning errors and retrieval misalignments.

Findings

Significant performance improvements over state-of-the-art methods.

Effective handling of ambiguous and multi-hop reasoning questions.

Robust across five diverse question answering datasets.

Abstract

In real practice, questions are typically complex and knowledge-intensive, requiring Large Language Models (LLMs) to recognize the multifaceted nature of the question and reason across multiple information sources. Iterative and adaptive retrieval, where LLMs decide when and what to retrieve based on their reasoning, has been shown to be a promising approach to resolve complex, knowledge-intensive questions. However, the performance of such retrieval frameworks is limited by the accumulation of reasoning errors and misaligned retrieval results. To overcome these limitations, we propose TreeRare (Syntax Tree-Guided Retrieval and Reasoning), a framework that utilizes syntax trees to guide information retrieval and reasoning for question answering. Following the principle of compositionality, TreeRare traverses the syntax tree in a bottom-up fashion, and in each node, it generates subcomponent-based queries and retrieves relevant passages to resolve localized uncertainty. A subcomponent question answering module then synthesizes these passages into concise, context-aware evidence. Finally, TreeRare aggregates the evidence across the tree to form a final answer. Experiments across five question answering datasets involving ambiguous or multi-hop reasoning demonstrate that TreeRare achieves substantial improvements over existing state-of-the-art methods.