A Context-Aware Computational Approach for Measuring Vocal Entrainment in Dyadic Conversations

TL;DR

This paper introduces a context-aware computational method using conformers and attention mechanisms to measure vocal entrainment in dyadic conversations, revealing its relevance to clinical assessments and social interaction analysis.

Contribution

It presents a novel approach combining conformers and cross-subject attention to quantify vocal entrainment, validated across diverse domains and clinical groups.

Findings

Successfully distinguished real from fake conversations.

Found significant correlation between entrainment measures and clinical scores.

Demonstrated robustness across gender and age groups.

Abstract

Vocal entrainment is a social adaptation mechanism in human interaction, knowledge of which can offer useful insights to an individual's cognitive-behavioral characteristics. We propose a context-aware approach for measuring vocal entrainment in dyadic conversations. We use conformers(a combination of convolutional network and transformer) for capturing both short-term and long-term conversational context to model entrainment patterns in interactions across different domains. Specifically we use cross-subject attention layers to learn intra- as well as inter-personal signals from dyadic conversations. We first validate the proposed method based on classification experiments to distinguish between real(consistent) and fake(inconsistent/shuffled) conversations. Experimental results on interactions involving individuals with Autism Spectrum Disorder also show evidence of a statistically-significant association between the introduced entrainment measure and clinical scores relevant to symptoms, including across gender and age groups.