HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

TL;DR

HippoRAG is a neurobiologically inspired retrieval framework for large language models that enhances knowledge integration and outperforms existing methods in multi-hop question answering, while being more efficient.

Contribution

Introduces HippoRAG, a novel hippocampal-inspired retrieval framework that improves knowledge integration in LLMs and outperforms state-of-the-art methods in efficiency and accuracy.

Findings

Outperforms state-of-the-art RAG methods by up to 20% in multi-hop QA

Single-step HippoRAG retrieval is 10-30 times cheaper and 6-13 times faster

Enables new scenarios beyond existing methods

Abstract

In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoiding catastrophic forgetting. Despite the impressive accomplishments, large language models (LLMs), even with retrieval-augmented generation (RAG), still struggle to efficiently and effectively integrate a large amount of new experiences after pre-training. In this work, we introduce HippoRAG, a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory to enable deeper and more efficient knowledge integration over new experiences. HippoRAG synergistically orchestrates LLMs, knowledge graphs, and the Personalized PageRank algorithm to mimic the different roles of neocortex and hippocampus in human memory. We compare HippoRAG with existing RAG methods on multi-hop question answering and show that our method outperforms the state-of-the-art methods remarkably, by up to 20%. Single-step retrieval with HippoRAG achieves comparable or better performance than iterative retrieval like IRCoT while being 10-30 times cheaper and 6-13 times faster, and integrating HippoRAG into IRCoT brings further substantial gains. Finally, we show that our method can tackle new types of scenarios that are out of reach of existing methods. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.