Defining and Quantifying Conversation Quality in Spontaneous Interactions

TL;DR

This paper introduces a new measure called perceived Conversation Quality to quantify spontaneous social interactions, supported by a questionnaire and analysis showing variability in annotator agreement, especially in low-quality conversations.

Contribution

It presents a novel perceived measure for spontaneous interactions and a questionnaire for assessing conversation quality at individual and group levels.

Findings

Naive annotators show less agreement on low-quality conversations.

The perceived Conversation Quality measure captures socio-dimensional aspects.

Group-level annotations have lower inter-annotator agreement.

Abstract

Social interactions in general are multifaceted and there exists a wide set of factors and events that influence them. In this paper, we quantify social interactions with a holistic viewpoint on individual experiences, particularly focusing on non-task-directed spontaneous interactions. To achieve this, we design a novel perceived measure, the perceived Conversation Quality, which intends to quantify spontaneous interactions by accounting for several socio-dimensional aspects of individual experiences. To further quantitatively study spontaneous interactions, we devise a questionnaire which measures the perceived Conversation Quality, at both the individual- and at the group- level. Using the questionnaire, we collected perceived annotations for conversation quality in a publicly available dataset using naive annotators. The results of the analysis performed on the distribution and the inter-annotator agreeability shows that naive annotators tend to agree less in cases of low conversation quality samples, especially while annotating for group-level conversation quality.