Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

TL;DR

This paper introduces the Hierarchical Aggregate Tree (HAT), a memory structure that enhances long-term reasoning in language models by efficiently aggregating dialogue context, leading to improved coherence and summary quality in multi-turn conversations.

Contribution

The paper presents a novel hierarchical memory structure, HAT, that enables recursive context aggregation and efficient long-term reasoning without exponential parameter increase.

Findings

HAT improves dialogue coherence

HAT enhances summary quality

Effective for multi-turn reasoning

Abstract

Large language models have limited context capacity, hindering reasoning over long conversations. We propose the Hierarchical Aggregate Tree memory structure to recursively aggregate relevant dialogue context through conditional tree traversals. HAT encapsulates information from children nodes, enabling broad coverage with depth control. We formulate finding best context as optimal tree traversal. Experiments show HAT improves dialog coherence and summary quality over baseline contexts, demonstrating the techniques effectiveness for multi turn reasoning without exponential parameter growth. This memory augmentation enables more consistent, grounded longform conversations from LLMs