Enhancing Retrieval-Augmented Generation for Electric Power Industry Customer Support

TL;DR

This paper presents a robust customer support system for the electric power industry that combines intent recognition, RAG Fusion, and reranking, significantly improving query handling and accuracy over baseline models.

Contribution

It introduces a novel combination of techniques—intent recognition, RAG Fusion, and reranking—for improved complex query handling in electric power customer support.

Findings

Query rewriting improves retrieval for non-standard terminology.

RAG Fusion enhances performance on vague, multi-faceted queries.

Reranking reduces hallucinations and irrelevant context inclusion.

Abstract

Many AI customer service systems use standard NLP pipelines or finetuned language models, which often fall short on ambiguous, multi-intent, or detail-specific queries. This case study evaluates recent techniques: query rewriting, RAG Fusion, keyword augmentation, intent recognition, and context reranking, for building a robust customer support system in the electric power domain. We compare vector-store and graph-based RAG frameworks, ultimately selecting the graph-based RAG for its superior performance in handling complex queries. We find that query rewriting improves retrieval for queries using non-standard terminology or requiring precise detail. RAG Fusion boosts performance on vague or multifaceted queries by merging multiple retrievals. Reranking reduces hallucinations by filtering irrelevant contexts. Intent recognition supports the decomposition of complex questions into more targeted sub-queries, increasing both relevance and efficiency. In contrast, keyword augmentation negatively impacts results due to biased keyword selection. Our final system combines intent recognition, RAG Fusion, and reranking to handle disambiguation and multi-source queries. Evaluated on both a GPT-4-generated dataset and a real-world electricity provider FAQ dataset, it achieves 97.9% and 89.6% accuracy respectively, substantially outperforming baseline RAG models.