Learning to Plan and Realize Separately for Open-Ended Dialogue Systems

TL;DR

This paper proposes a two-phase approach for open-ended dialogue systems, separating planning and realization to improve conversational quality over traditional end-to-end models.

Contribution

It introduces a novel decoupled framework for dialogue generation, explicitly modeling human-like planning and realization processes.

Findings

Decoupling planning and realization outperforms end-to-end models in evaluations.

Human and automated assessments favor the two-phase approach.

The method enhances the naturalness and coherence of generated responses.

Abstract

Achieving true human-like ability to conduct a conversation remains an elusive goal for open-ended dialogue systems. We posit this is because extant approaches towards natural language generation (NLG) are typically construed as end-to-end architectures that do not adequately model human generation processes. To investigate, we decouple generation into two separate phases: planning and realization. In the planning phase, we train two planners to generate plans for response utterances. The realization phase uses response plans to produce an appropriate response. Through rigorous evaluations, both automated and human, we demonstrate that decoupling the process into planning and realization performs better than an end-to-end approach.