Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversations

TL;DR

This paper introduces a real-time question identification and answer generation system for customer contact centers that improves efficiency by accurately distinguishing FAQs from complex queries and providing rapid responses, reducing handling times and operational costs.

Contribution

It presents a novel decision support system that combines FAQ retrieval and RAG-based answer generation, including an automated workflow for FAQ detection from transcripts, enhancing real-time customer support.

Findings

System responds within 2 seconds in live deployment.

Reduces average handling time and operational costs.

Improves accuracy of query interpretation in real-time conversations.

Abstract

In customer contact centers, human agents often struggle with long average handling times (AHT) due to the need to manually interpret queries and retrieve relevant knowledge base (KB) articles. While retrieval augmented generation (RAG) systems using large language models (LLMs) have been widely adopted in industry to assist with such tasks, RAG faces challenges in real-time conversations, such as inaccurate query formulation and redundant retrieval of frequently asked questions (FAQs). To address these limitations, we propose a decision support system that can look beyond RAG by first identifying customer questions in real time. If the query matches an FAQ, the system retrieves the answer directly from the FAQ database; otherwise, it generates answers via RAG. Our approach reduces reliance on manual queries, providing responses to agents within 2 seconds. Deployed in AI-powered human-agent assist solution at Minerva CQ, this system improves efficiency, reduces AHT, and lowers operational costs. We also introduce an automated LLM-agentic workflow to identify FAQs from historical transcripts when no predefined FAQs exist.