Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction

TL;DR

This paper introduces a two-stage CRC framework that enhances goal-oriented proactive dialogue systems by reflecting on and correcting inconsistencies between generated responses and dialogue contexts, improving response coherence.

Contribution

The paper proposes a model-agnostic CRC framework with reflection and correction stages to improve response consistency in dialogue systems, addressing a gap in previous research.

Findings

Significant improvement in response consistency across multiple models.

Effective across various datasets and model architectures.

Enhances dialogue system reliability and user experience.

Abstract

Goal-oriented proactive dialogue systems are designed to guide user conversations seamlessly towards specific objectives by planning a goal-oriented path. However, previous research has focused predominantly on optimizing these paths while neglecting the inconsistencies that may arise between generated responses and dialogue contexts, including user profiles, dialogue history, domain knowledge, and subgoals. To address this issue, we introduce a model-agnostic two-stage Consistency Reflection and Correction (CRC) framework. Specifically, in the consistency reflection stage, the model is prompted to reflect on the discrepancies between generated responses and dialogue contexts, identifying inconsistencies and suggesting possible corrections. In the consistency correction stage, the model generates responses that are more consistent with the dialogue context based on these reflection results. We conducted experiments on various model architectures with different parameter sizes, including encoder-decoder models (BART, T5) and decoder-only models (GPT-2, DialoGPT, Phi3, Mistral and LLaMA3), and the experimental results on three datasets demonstrate that our CRC framework significantly improves the consistency between generated responses and dialogue contexts.