A Similarity Measure for Comparing Conversational Dynamics

TL;DR

This paper introduces a new similarity measure for comparing the overall dynamics of conversations, aiming to improve analysis and evaluation of conversational quality and agent performance.

Contribution

It proposes a novel metric for assessing conversation dynamics and validates its robustness and sensitivity, providing a tool for holistic conversation analysis.

Findings

The measure captures differences in conversational dynamics effectively.

It reveals insights into the role of situational power in online conversations.

The metric is robust across various topics and conversation types.

Abstract

The quality of a conversation goes beyond the individual quality of each reply, and instead emerges from how these combine into interactional dynamics that give the conversation its distinctive overall "shape". However, there is no robust automated method for comparing conversations in terms of their overall dynamics. Such methods could enhance the analysis of conversational data and help evaluate conversational agents more holistically. In this work, we introduce a similarity measure for comparing conversations with respect to their dynamics. We design a validation procedure for testing the robustness of the metric in capturing differences in conversation dynamics and for assessing its sensitivity to the topic of the conversations. To illustrate the measure's utility, we use it to analyze conversational dynamics in a large online community, bringing new insights into the role of situational power in conversations.