Redefining Proactivity for Information Seeking Dialogue
Jing Yang Lee, Seokhwan Kim, Kartik Mehta, Jiun-Yu Kao, Yu-Hsiang Lin,, Arpit Gupta

TL;DR
This paper redefines proactivity in information-seeking dialogue by focusing on responses that actively introduce new, relevant information to engage users and sustain conversations, supported by a new dataset and evaluation metrics.
Contribution
It introduces a new definition of proactivity emphasizing engagement, creates a dataset of 2,000 conversations, and proposes novel Chain-of-Thought prompts that significantly improve response proactiveness.
Findings
High correlation between automatic metrics and human judgment
Proposed prompts outperform standard prompts by up to 90% in zero-shot settings
New dataset enables better evaluation of proactive responses
Abstract
Information-Seeking Dialogue (ISD) agents aim to provide accurate responses to user queries. While proficient in directly addressing user queries, these agents, as well as LLMs in general, predominantly exhibit reactive behavior, lacking the ability to generate proactive responses that actively engage users in sustained conversations. However, existing definitions of proactive dialogue in this context do not focus on how each response actively engages the user and sustains the conversation. Hence, we present a new definition of proactivity that focuses on enhancing the `proactiveness' of each generated response via the introduction of new information related to the initial query. To this end, we construct a proactive dialogue dataset comprising 2,000 single-turn conversations, and introduce several automatic metrics to evaluate response `proactiveness' which achieved high correlation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsSpeech and dialogue systems
MethodsFocus
