Mixture of Demonstrations for Textual Graph Understanding and Question Answering

TL;DR

This paper introduces MixDemo, a novel framework that improves question answering over textual graphs by selecting high-quality demonstrations with a Mixture-of-Experts mechanism and reducing noise with a query-specific encoder, leading to superior performance.

Contribution

We propose MixDemo, combining a Mixture-of-Experts demonstration selection and a query-specific graph encoder to enhance reasoning and accuracy in GraphRAG-based question answering.

Findings

Significant performance improvements over existing methods.

Effective demonstration selection via Mixture-of-Experts.

Reduced noise in retrieved subgraphs improves reasoning.

Abstract

Textual graph-based retrieval-augmented generation (GraphRAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) in domain-specific question answering. While existing approaches primarily focus on zero-shot GraphRAG, selecting high-quality demonstrations is crucial for improving reasoning and answer accuracy. Furthermore, recent studies have shown that retrieved subgraphs often contain irrelevant information, which can degrade reasoning performance. In this paper, we propose MixDemo, a novel GraphRAG framework enhanced with a Mixture-of-Experts (MoE) mechanism for selecting the most informative demonstrations under diverse question contexts. To further reduce noise in the retrieved subgraphs, we introduce a query-specific graph encoder that selectively attends to information most relevant to the query. Extensive experiments across multiple textual graph benchmarks show that MixDemo significantly outperforms existing methods.