DTCRS: Dynamic Tree Construction for Recursive Summarization

TL;DR

DTCRS is a dynamic method for constructing hierarchical summaries tailored to question types, reducing redundancy and improving efficiency in multi-step reasoning tasks with large documents.

Contribution

It introduces a question-aware, dynamic summary tree construction approach that reduces redundancy and enhances relevance for recursive summarization in QA tasks.

Findings

Reduces summary tree construction time significantly.

Improves relevance of summaries to questions across three QA tasks.

Analyzes applicability of recursive summarization to different question types.

Abstract

Retrieval-Augmented Generation (RAG) mitigates the hallucination problem of Large Language Models (LLMs) by incorporating external knowledge. Recursive summarization constructs a hierarchical summary tree by clustering text chunks, integrating information from multiple parts of a document to provide evidence for abstractive questions involving multi-step reasoning. However, summary trees often contain a large number of redundant summary nodes, which not only increase construction time but may also negatively impact question answering. Moreover, recursive summarization is not suitable for all types of questions. We introduce DTCRS, a method that dynamically generates summary trees based on document structure and query semantics. DTCRS determines whether a summary tree is necessary by analyzing the question type. It then decomposes the question and uses the embeddings of sub-questions as initial cluster centers, reducing redundant summaries while improving the relevance between summaries and the question. Our approach significantly reduces summary tree construction time and achieves substantial improvements across three QA tasks. Additionally, we investigate the applicability of recursive summarization to different question types, providing valuable insights for future research.