Persuasive Dialogue Understanding: the Baselines and Negative Results

TL;DR

This paper evaluates Transformer-based models with CRF for persuasive strategy recognition in dialogues, revealing their limitations and negative results, and highlighting the importance of model choice and sequence modeling in this task.

Contribution

The study demonstrates the limitations of Transformer+CRF models for persuasive strategy recognition and provides insights into effective sequence modeling in dialogue understanding.

Findings

Transformer+CRF models do not outperform baselines.

CRF cannot effectively capture persuasive label dependencies.

LSTM outperforms Transformer in capturing sequential information.

Abstract

Persuasion aims at forming one's opinion and action via a series of persuasive messages containing persuader's strategies. Due to its potential application in persuasive dialogue systems, the task of persuasive strategy recognition has gained much attention lately. Previous methods on user intent recognition in dialogue systems adopt recurrent neural network (RNN) or convolutional neural network (CNN) to model context in conversational history, neglecting the tactic history and intra-speaker relation. In this paper, we demonstrate the limitations of a Transformer-based approach coupled with Conditional Random Field (CRF) for the task of persuasive strategy recognition. In this model, we leverage inter- and intra-speaker contextual semantic features, as well as label dependencies to improve the recognition. Despite extensive hyper-parameter optimizations, this architecture fails to outperform the baseline methods. We observe two negative results. Firstly, CRF cannot capture persuasive label dependencies, possibly as strategies in persuasive dialogues do not follow any strict grammar or rules as the cases in Named Entity Recognition (NER) or part-of-speech (POS) tagging. Secondly, the Transformer encoder trained from scratch is less capable of capturing sequential information in persuasive dialogues than Long Short-Term Memory (LSTM). We attribute this to the reason that the vanilla Transformer encoder does not efficiently consider relative position information of sequence elements.