UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG

TL;DR

ULTRAG introduces a novel framework for Knowledge Graph question answering that leverages off-the-shelf neural modules, enabling state-of-the-art performance without retraining large language models.

Contribution

It presents a new retrieval framework for Knowledge Graphs that achieves high accuracy and scalability without retraining LLMs or neural query executors.

Findings

ULTRAG outperforms existing KG-RAG methods in accuracy.

It enables LLMs to handle Wikidata-scale graphs efficiently.

The approach achieves comparable or lower computational costs.

Abstract

Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon often known as hallucination). Retrieval augmented generation (RAG) tries to reduce factual errors by identifying information in a knowledge corpus and putting it in the context window of the model. While this approach is well-established for document-structured data, it is non-trivial to adapt it for Knowledge Graphs (KGs), especially for queries that require multi-node/multi-hop reasoning on graphs. We introduce ULTRAG, a general framework for retrieving information from Knowledge Graphs that shifts away from classical RAG. By endowing LLMs with off-the-shelf neural query executing modules, we highlight how readily available language models can achieve state-of-the-art results on Knowledge Graph Question Answering (KGQA) tasks without any retraining of the LLM or executor involved. In our experiments, ULTRAG achieves better performance when compared to state-of-the-art KG-RAG solutions, and it enables language models to interface with Wikidata-scale graphs (116M entities, 1.6B relations) at comparable or lower costs.