Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

TL;DR

This paper introduces a novel multi-task learning approach for dialogue coherence assessment that does not require explicit dialogue act labels during evaluation, significantly improving performance on benchmark datasets.

Contribution

It proposes using dialogue act prediction as an auxiliary task to enhance utterance representations for coherence assessment without relying on dialogue act labels at test time.

Findings

Over 20 accuracy points improvement on DailyDialogue corpus

Comparable performance to state-of-the-art on SwitchBoard corpus

Effective in ranking dialogue coherence without explicit dialogue act labels

Abstract

Recent dialogue coherence models use the coherence features designed for monologue texts, e.g. nominal entities, to represent utterances and then explicitly augment them with dialogue-relevant features, e.g., dialogue act labels. It indicates two drawbacks, (a) semantics of utterances is limited to entity mentions, and (b) the performance of coherence models strongly relies on the quality of the input dialogue act labels. We address these issues by introducing a novel approach to dialogue coherence assessment. We use dialogue act prediction as an auxiliary task in a multi-task learning scenario to obtain informative utterance representations for coherence assessment. Our approach alleviates the need for explicit dialogue act labels during evaluation. The results of our experiments show that our model substantially (more than 20 accuracy points) outperforms its strong competitors on the DailyDialogue corpus, and performs on par with them on the SwitchBoard corpus for ranking dialogues concerning their coherence.