Coherence Models for Dialogue

TL;DR

This paper enhances dialogue coherence modeling by integrating intent information into the entity grid approach, significantly improving performance on coherence assessment tasks across multiple datasets.

Contribution

It introduces a novel augmentation of the entity grid with dialogue intent structure, addressing a key gap in previous coherence models.

Findings

Models with intent information outperform original entity grid models.

Enhanced models show improved accuracy in coherence tasks.

Intent integration is crucial for effective dialogue coherence modeling.

Abstract

Coherence across multiple turns is a major challenge for state-of-the-art dialogue models. Arguably the most successful approach to automatically learning text coherence is the entity grid, which relies on modelling patterns of distribution of entities across multiple sentences of a text. Originally applied to the evaluation of automatic summaries and the news genre, among its many extensions, this model has also been successfully used to assess dialogue coherence. Nevertheless, both the original grid and its extensions do not model intents, a crucial aspect that has been studied widely in the literature in connection to dialogue structure. We propose to augment the original grid document representation for dialogue with the intentional structure of the conversation. Our models outperform the original grid representation on both text discrimination and insertion, the two main standard tasks for coherence assessment across three different dialogue datasets, confirming that intents play a key role in modelling dialogue coherence.