Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance

TL;DR

This paper introduces a user-oriented proactive chatbot that uses critic-guided training and curriculum learning to improve engagement and personalization in open-domain dialogues, addressing limitations of existing LLM-based systems.

Contribution

The paper proposes a novel critic-guided training framework and curriculum learning approach to enhance user-oriented proactivity in open-domain dialogue systems, supported by a diverse user background dataset.

Findings

Improved user engagement and satisfaction in dialogues.

Enhanced proactivity and personalization in chatbot responses.

Effective training method applicable across different LLMs.

Abstract

Open-domain dialogue systems aim to generate natural and engaging conversations, providing significant practical value in real applications such as social robotics and personal assistants. The advent of large language models (LLMs) has greatly advanced this field by improving context understanding and conversational fluency. However, existing LLM-based dialogue systems often fall short in proactively understanding the user's chatting preferences and guiding conversations toward user-centered topics. This lack of user-oriented proactivity can lead users to feel unappreciated, reducing their satisfaction and willingness to continue the conversation in human-computer interactions. To address this issue, we propose a User-oriented Proactive Chatbot (UPC) to enhance the user-oriented proactivity. Specifically, we first construct a critic to evaluate this proactivity inspired by the LLM-as-a-judge strategy. Given the scarcity of high-quality training data, we then employ the critic to guide dialogues between the chatbot and user agents, generating a corpus with enhanced user-oriented proactivity. To ensure the diversity of the user backgrounds, we introduce the ISCO-800, a diverse user background dataset for constructing user agents. Moreover, considering the communication difficulty varies among users, we propose an iterative curriculum learning method that trains the chatbot from easy-to-communicate users to more challenging ones, thereby gradually enhancing its performance. Experiments demonstrate that our proposed training method is applicable to different LLMs, improving user-oriented proactivity and attractiveness in open-domain dialogues.