Adaptive Multi-Agent Response Refinement in Conversational Systems

TL;DR

This paper introduces an adaptive multi-agent response refinement framework for conversational AI, where specialized agents collaboratively improve responses by focusing on factuality, personalization, and coherence, leading to better conversational quality.

Contribution

It presents a novel multi-agent system with dynamic communication for response refinement, addressing limitations of single-model approaches in conversational AI.

Findings

Outperforms baseline methods on challenging datasets

Improves factual accuracy, personalization, and coherence

Effective in knowledge and persona-driven conversations

Abstract

Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or specific knowledge. In real-life settings, it is impractical to rely on users to detect these errors and request a new response. One way to address this problem is to refine the response before returning it to the user. While existing approaches focus on refining responses within a single LLM, this method struggles to consider diverse aspects needed for effective conversations. In this work, we propose refining responses through a multi-agent framework, where each agent is assigned a specific role for each aspect. We focus on three key aspects crucial to conversational quality: factuality, personalization, and coherence. Each agent is responsible for reviewing and refining one of these aspects, and their feedback is then merged to improve the overall response. To enhance collaboration among them, we introduce a dynamic communication strategy. Instead of following a fixed sequence of agents, our approach adaptively selects and coordinates the most relevant agents based on the specific requirements of each query. We validate our framework on challenging conversational datasets, demonstrating that ours significantly outperforms relevant baselines, particularly in tasks involving knowledge or user's persona, or both.