Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling

TL;DR

This paper introduces BiDeN, a bidirectional dialogue encoder that explicitly captures past and future contexts, improving multi-turn dialogue understanding across various tasks.

Contribution

The paper proposes BiDeN, a novel universal dialogue encoder that effectively models temporal context in dialogues, surpassing traditional PrLM-based methods.

Findings

BiDeN outperforms existing models on multiple dialogue tasks.

BiDeN effectively captures both past and future dialogue contexts.

Experimental results demonstrate BiDeN's universality and effectiveness.

Abstract

Multi-turn dialogue modeling as a challenging branch of natural language understanding (NLU), aims to build representations for machines to understand human dialogues, which provides a solid foundation for multiple downstream tasks. Recent studies of dialogue modeling commonly employ pre-trained language models (PrLMs) to encode the dialogue history as successive tokens, which is insufficient in capturing the temporal characteristics of dialogues. Therefore, we propose Bidirectional Information Decoupling Network (BiDeN) as a universal dialogue encoder, which explicitly incorporates both the past and future contexts and can be generalized to a wide range of dialogue-related tasks. Experimental results on datasets of different downstream tasks demonstrate the universality and effectiveness of our BiDeN.