Generic Dependency Modeling for Multi-Party Conversation

TL;DR

This paper introduces a generic dependency modeling framework for multi-party conversations using dependency parsing, enhancing Transformer models' performance by encoding utterance dependencies through ReDE.

Contribution

It proposes a novel dependency encoding method, ReDE, integrated into Transformers to improve modeling of multi-party conversational dependencies.

Findings

Boosts performance of Transformer-based models on four benchmarks

Achieves comparable or superior results to state-of-the-art methods

Demonstrates effectiveness of dependency encoding in conversation modeling

Abstract

To model the dependencies between utterances in multi-party conversations, we propose a simple and generic framework based on the dependency parsing results of utterances. Particularly, we present an approach to encoding the dependencies in the form of relative dependency encoding (ReDE) and illustrate how to implement it in Transformers by modifying the computation of self-attention. Experimental results on four multi-party conversation benchmarks show that this framework successfully boosts the general performance of two Transformer-based language models and leads to comparable or even superior performance compared to the state-of-the-art methods. The codes are available at https://github.com/shenwzh3/ReDE.