Analyzing Team Performance with Embeddings from Multiparty Dialogues

TL;DR

This paper investigates how embeddings derived from multiparty dialogues, based on dialogue acts, sentiment, and syntactic entrainment, can predict team performance across different teamwork phases, highlighting the importance of dialogue act and sentiment features.

Contribution

It introduces a method to predict team performance from dialogue embeddings and analyzes the varying utility of features across teamwork phases.

Findings

Dialogue act and sentiment embeddings effectively predict team performance early on.

Syntactic entrainment embeddings are less effective for early-stage prediction.

Different features have varying utility depending on the teamwork phase.

Abstract

Good communication is indubitably the foundation of effective teamwork. Over time teams develop their own communication styles and often exhibit entrainment, a conversational phenomena in which humans synchronize their linguistic choices. This paper examines the problem of predicting team performance from embeddings learned from multiparty dialogues such that teams with similar conflict scores lie close to one another in vector space. Embeddings were extracted from three types of features: 1) dialogue acts 2) sentiment polarity 3) syntactic entrainment. Although all of these features can be used to effectively predict team performance, their utility varies by the teamwork phase. We separate the dialogues of players playing a cooperative game into stages: 1) early (knowledge building) 2) middle (problem-solving) and 3) late (culmination). Unlike syntactic entrainment, both dialogue act and sentiment embeddings are effective for classifying team performance, even during the initial phase. This finding has potential ramifications for the development of conversational agents that facilitate teaming.