Augmenting Textual Generation via Topology Aware Retrieval

TL;DR

This paper introduces a topology-aware retrieval framework for text generation that leverages graph relationships to improve the relevance and accuracy of generated content by guiding retrieval and integration of external texts.

Contribution

It proposes a novel method that incorporates topological graph information into retrieval-augmented generation, enhancing the relevance of retrieved texts for better language model outputs.

Findings

Topological relationships improve retrieval relevance.

The framework enhances text generation quality.

Empirical results validate the effectiveness of topology-aware retrieval.

Abstract

Despite the impressive advancements of Large Language Models (LLMs) in generating text, they are often limited by the knowledge contained in the input and prone to producing inaccurate or hallucinated content. To tackle these issues, Retrieval-augmented Generation (RAG) is employed as an effective strategy to enhance the available knowledge base and anchor the responses in reality by pulling additional texts from external databases. In real-world applications, texts are often linked through entities within a graph, such as citations in academic papers or comments in social networks. This paper exploits these topological relationships to guide the retrieval process in RAG. Specifically, we explore two kinds of topological connections: proximity-based, focusing on closely connected nodes, and role-based, which looks at nodes sharing similar subgraph structures. Our empirical research confirms their relevance to text relationships, leading us to develop a Topology-aware Retrieval-augmented Generation framework. This framework includes a retrieval module that selects texts based on their topological relationships and an aggregation module that integrates these texts into prompts to stimulate LLMs for text generation. We have curated established text-attributed networks and conducted comprehensive experiments to validate the effectiveness of this framework, demonstrating its potential to enhance RAG with topological awareness.