Inconsistent dialogue responses and how to recover from them
Mian Zhang, Lifeng Jin, Linfeng Song, Haitao Mi, Dong Yu

TL;DR
This paper introduces a new dataset and tasks for detecting and resolving inconsistencies in dialogue responses, highlighting current models' strengths and weaknesses in maintaining conversational consistency.
Contribution
It develops a comprehensive dataset covering the entire inconsistency lifecycle and proposes tasks that advance research on dialogue consistency assessment and improvement.
Findings
Dataset improves inconsistency detection and resolution
Large language models excel at resolution but struggle with detection
Study advances understanding of dialogue consistency challenges
Abstract
One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem. In this work, we study methods to assess and bolster utterance consistency of chat systems. A dataset is first developed for studying the inconsistencies, where inconsistent dialogue responses, explanations of the inconsistencies, and recovery utterances are authored by annotators. This covers the life span of inconsistencies, namely introduction, understanding, and resolution. Building on this, we introduce a set of tasks centered on dialogue consistency, specifically focused on its detection and resolution. Our experimental findings indicate that our dataset significantly helps the progress in identifying and resolving conversational inconsistencies, and current popular large language models like ChatGPT which are good at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Speech and dialogue systems · Misinformation and Its Impacts
MethodsSparse Evolutionary Training
