Improving the Generalizability of Collaborative Dialogue Analysis with Multi-Feature Embeddings

TL;DR

This paper introduces a multi-feature embedding method that enhances the generalizability of conflict prediction models in collaborative dialogues, especially under resource scarcity and domain shifts, by integrating lexical, dialogue act, and sentiment features.

Contribution

The paper proposes MFeEmb, a novel multi-feature embedding that improves conflict prediction across domains by leveraging textual, structural, and semantic dialogue features.

Findings

MFeEmb outperforms baseline models in domain adaptation tasks.

Incorporating dialogue acts and sentiment features reduces performance loss.

MFeEmb is effective for meta-pretraining in few-shot learning scenarios.

Abstract

Conflict prediction in communication is integral to the design of virtual agents that support successful teamwork by providing timely assistance. The aim of our research is to analyze discourse to predict collaboration success. Unfortunately, resource scarcity is a problem that teamwork researchers commonly face since it is hard to gather a large number of training examples. To alleviate this problem, this paper introduces a multi-feature embedding (MFeEmb) that improves the generalizability of conflict prediction models trained on dialogue sequences. MFeEmb leverages textual, structural, and semantic information from the dialogues by incorporating lexical, dialogue acts, and sentiment features. The use of dialogue acts and sentiment features reduces performance loss from natural distribution shifts caused mainly by changes in vocabulary. This paper demonstrates the performance of MFeEmb on domain adaptation problems in which the model is trained on discourse from one task domain and applied to predict team performance in a different domain. The generalizability of MFeEmb is quantified using the similarity measure proposed by Bontonou et al. (2021). Our results show that MFeEmb serves as an excellent domain-agnostic representation for meta-pretraining a few-shot model on collaborative multiparty dialogues.