Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models

TL;DR

This paper introduces OKH-RAG, a hypergraph-based retrieval method that explicitly models the order of evidence, improving reasoning in order-sensitive tasks over traditional permutation-invariant approaches.

Contribution

It proposes a novel order-aware hypergraph retrieval framework that captures the sequence of interactions, enhancing reasoning capabilities in language models.

Findings

OKH-RAG outperforms permutation-invariant baselines on order-sensitive tasks.

Modeling interaction order improves reasoning accuracy.

A learned transition model infers precedence without explicit temporal labels.

Abstract

Retrieval-augmented generation (RAG) enhances large language models by grounding outputs in retrieved knowledge. However, existing RAG methods including graph- and hypergraph-based approaches treat retrieved evidence as an unordered set, implicitly assuming permutation invariance. This assumption is misaligned with many real-world reasoning tasks, where outcomes depend not only on which interactions occur, but also on the order in which they unfold. We propose Order-Aware Knowledge Hypergraph RAG (OKH-RAG), which treats order as a first-class structural property. OKH-RAG represents knowledge as higher-order interactions within a hypergraph augmented with precedence structure, and reformulates retrieval as sequence inference over hyperedges. Instead of selecting independent facts, it recovers coherent interaction trajectories that reflect underlying reasoning processes. A learned transition model infers precedence directly from data without requiring explicit temporal supervision. We evaluate OKH-RAG on order-sensitive question answering and explanation tasks, including tropical cyclone and port operation scenarios. OKH-RAG consistently outperforms permutation-invariant baselines, and ablations show that these gains arise specifically from modeling interaction order. These results highlight a key limitation of set-based retrieval: effective reasoning requires not only retrieving relevant evidence, but organizing it into structured sequences.