How Did We Get Here? Summarizing Conversation Dynamics

TL;DR

This paper introduces the task of summarizing conversation dynamics, creating a dataset of human-written summaries, and demonstrates that these summaries improve the accuracy and speed of predicting conversation derailment into toxic behavior.

Contribution

It presents a new dataset of conversation dynamic summaries and shows their effectiveness in enhancing both human and automated forecasting of conversation outcomes.

Findings

Summaries enable humans to predict conversation derailment three times faster.

Automated systems are more accurate when using summaries rather than transcripts.

Summaries help capture the trajectory of conversations beyond factual content.

Abstract

Throughout a conversation, the way participants interact with each other is in constant flux: their tones may change, they may resort to different strategies to convey their points, or they might alter their interaction patterns. An understanding of these dynamics can complement that of the actual facts and opinions discussed, offering a more holistic view of the trajectory of the conversation: how it arrived at its current state and where it is likely heading. In this work, we introduce the task of summarizing the dynamics of conversations, by constructing a dataset of human-written summaries, and exploring several automated baselines. We evaluate whether such summaries can capture the trajectory of conversations via an established downstream task: forecasting whether an ongoing conversation will eventually derail into toxic behavior. We show that they help both humans and automated systems with this forecasting task. Humans make predictions three times faster, and with greater confidence, when reading the summaries than when reading the transcripts. Furthermore, automated forecasting systems are more accurate when constructing, and then predicting based on, summaries of conversation dynamics, compared to directly predicting on the transcripts.