HyperbolicRAG: Enhancing Retrieval-Augmented Generation with Hyperbolic Representations

TL;DR

HyperbolicRAG introduces hyperbolic geometry into retrieval-augmented generation to better model hierarchical knowledge, improving reasoning and accuracy over existing Euclidean-based methods.

Contribution

It presents a novel hyperbolic embedding framework for RAG that captures hierarchy and semantics simultaneously, with new regularization and fusion mechanisms.

Findings

Outperforms standard RAG and graph-based baselines on QA benchmarks.

Effectively models hierarchical relationships in knowledge graphs.

Enhances reasoning capabilities of language models.

Abstract

Retrieval-augmented generation (RAG) enables large language models (LLMs) to access external knowledge, helping mitigate hallucinations and enhance domain-specific expertise. Graph-based RAG enhances structural reasoning by introducing explicit relational organization that enables information propagation across semantically connected text units. However, these methods typically rely on Euclidean embeddings that capture semantic similarity but lack a geometric notion of hierarchical depth, limiting their ability to represent abstraction relationships inherent in complex knowledge graphs. To capture both fine-grained semantics and global hierarchy, we propose HyperbolicRAG, a retrieval framework that integrates hyperbolic geometry into graph-based RAG. HyperbolicRAG introduces three key designs: (1) a depth-aware representation learner that embeds nodes within a shared Poincare manifold to align semantic similarity with hierarchical containment, (2) an unsupervised contrastive regularization that enforces geometric consistency across abstraction levels, and (3) a mutual-ranking fusion mechanism that jointly exploits retrieval signals from Euclidean and hyperbolic spaces, emphasizing cross-space agreement during inference. Extensive experiments across multiple QA benchmarks demonstrate that HyperbolicRAG outperforms competitive baselines, including both standard RAG and graph-augmented baselines.