DEMO: Reframing Dialogue Interaction with Fine-grained Element Modeling

TL;DR

This paper introduces DEMO, a new benchmark and task for detailed dialogue element modeling, aiming to improve the understanding, generation, and assessment of dialogue systems by focusing on comprehensive dialogue stages and elements.

Contribution

It proposes a novel dialogue element modeling task and benchmark, DEMO, along with a DEMO agent that enhances LLMs' ability to model dialogue elements through imitation learning.

Findings

Current LLMs show significant potential for improvement in dialogue element modeling.

The DEMO agent outperforms baseline models in both in-domain and out-of-domain tasks.

Extensive experiments validate the effectiveness of the DEMO benchmark and modeling approach.

Abstract

Large language models (LLMs) enabled dialogue systems have become one of the central modes in human-machine interaction, which bring about vast amounts of conversation logs and increasing demand for dialogue generation. The dialogue's life-cycle spans from $\textit{Prelude}$ through $\textit{Interlocution}$ to $\textit{Epilogue}$, encompassing rich dialogue elements. Despite large volumes of dialogue-related studies, there is a lack of systematic investigation into the dialogue stages to frame benchmark construction that covers comprehensive dialogue elements. This hinders the precise modeling, generation and assessment of LLMs-based dialogue systems. To bridge this gap, in this paper, we introduce a new research task--$\textbf{D}$ialogue $\textbf{E}$lement $\textbf{MO}$deling, including $\textit{Element Awareness}$ and $\textit{Dialogue Agent Interaction}$, and propose a novel benchmark, $\textbf{DEMO}$, designed for a comprehensive dialogue modeling and assessment. On this basis, we further build the DEMO agent with the adept ability to model dialogue elements via imitation learning. Extensive experiments on DEMO indicate that current representative LLMs still have considerable potential for enhancement, and our DEMO agent performs well in both dialogue element modeling and out-of-domain tasks.