CausalDialogue: Modeling Utterance-level Causality in Conversations

TL;DR

This paper introduces CausalDialogue, a new dataset and causality-aware training method for neural conversation models, aiming to improve dialogue naturalness by modeling utterance-level causality.

Contribution

It presents a novel causality-enhanced training approach called ExMATE and a new dataset with causal pairs, advancing dialogue generation research.

Findings

Causality-inspired loss improves response diversity

ExMATE enhances model agility in dialogue generation

Room remains for human-level quality in causality-aware models

Abstract

Despite their widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans. In this research, we examine user utterances as causes and generated responses as effects, recognizing that changes in a cause should produce a different effect. To further explore this concept, we have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. Our analysis reveals that traditional loss functions struggle to effectively incorporate the DAG structure, leading us to propose a causality-enhanced method called Exponential Maximum Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models. To evaluate the needs of considering causality in dialogue generation, we built a comprehensive benchmark on CausalDialogue dataset using different models, inference, and training methods. Through experiments, we find that a causality-inspired loss like ExMATE can improve the diversity and agility of conventional loss function and there is still room for improvement to reach human-level quality on this new dataset.