STAR: Semantic-Tuned and Tail-Adaptive Retriever for Graph-Augmented Generation

TL;DR

The paper introduces STAR, a novel retriever for Graph-Augmented Generation that addresses semantic and long-tail biases, improving multi-hop question answering performance with innovative learning paradigms.

Contribution

STAR combines token-level interaction and path-weighted contrastive learning to mitigate biases and enhance retrieval in GraphRAG systems.

Findings

STAR achieves 1.8% average retrieval performance gain.

STAR improves LLM QA performance by 2.2%.

Extensive experiments validate STAR's effectiveness.

Abstract

To augment Large Language Models (LLMs) for multi-hop question answering, a mainstream solution within Graph Retrieval Augmented Generation (GraphRAG) leverages lightweight retrievers to efficiently extract information from a given Knowledge Graph (KG). However, existing methods often overlook the inherent challenge of sparse semantic information in graphs. Specifically, our experiments reveal that these methods produce biased retrieval Semantic Shortcut Bias and Long-Tail Path Bias, leading to inadequate semantic modeling and limited GraphRAG effectiveness. To address these issues, we propose STAR, a semantic-tuned and tail-adaptive retriever for GraphRAG. STAR integrates two key learning paradigms: token-level interaction learning and path-weighted contrastive learning. The former employs a cross-attention architecture and a hard path mining mechanism to jointly model the query and path, thereby mitigating the Semantic Shortcut Bias. The latter introduces a tailored contrastive learning objective that utilizes tail-adaptive path weighting, designed to optimize the training process and ease the Long-Tail Path Bias. Extensive experiments demonstrate that STAR consistently outperforms baselines, achieving average retrieval performance gains of 1.8\% and LLM QA performance improvements of 2.2\% across all benchmark datasets. Our code is available at https://anonymous.4open.science/r/STAR-C583.