Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension

TL;DR

This paper introduces a novel channel-aware decoupling network that enhances multi-turn dialogue comprehension by explicitly modeling speaker roles and utterance interactions, significantly improving performance over existing pre-trained language models.

Contribution

The paper proposes a decoupling mechanism in Transformer-based PrLMs to better capture hierarchical dialogue features, addressing limitations of sequential dialogue modeling.

Findings

Achieves state-of-the-art results on four benchmark datasets.

Substantially improves baseline PrLM performance.

Effectively models speaker roles and utterance dependencies.

Abstract

Training machines to understand natural language and interact with humans is one of the major goals of artificial intelligence. Recent years have witnessed an evolution from matching networks to pre-trained language models (PrLMs). In contrast to the plain-text modeling as the focus of the PrLMs, dialogue texts involve multiple speakers and reflect special characteristics such as topic transitions and structure dependencies between distant utterances. However, the related PrLM models commonly represent dialogues sequentially by processing the pairwise dialogue history as a whole. Thus the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose compositional learning for holistic interaction across the utterances beyond the sequential contextualization from PrLMs, in order to capture the utterance-aware and speaker-aware representations entailed in a dialogue history. We decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, and two speaker roles (i.e., utterances of sender and utterances of the receiver), respectively. In addition, we employ domain-adaptive training strategies to help the model adapt to the dialogue domains. Experimental results show that our method substantially boosts the strong PrLM baselines in four public benchmark datasets, achieving new state-of-the-art performance over previous methods.