Multimodal Quantitative Measures for Multiparty Behaviour Evaluation
Ojas Shirekar, Wim Pouw, Chenxu Hao, Vrushank Phadnis, Thabo Beeler, Chirag Raman

TL;DR
This paper introduces a comprehensive framework for objectively evaluating multiparty social behavior in digital humans using three novel metrics that analyze synchrony, timing, and structural similarity in skeletal motion data.
Contribution
It presents a unified, intervention-driven evaluation framework with three complementary metrics, validated through perturbation experiments and perception studies, advancing the assessment of social behavior in digital agents.
Findings
Metrics detect changes due to gesture dampening, delays, and pitch flattening.
Metrics reveal predictable shifts in social coordination measures.
The framework provides orthogonal insights into spatial, temporal, and behavioral aspects.
Abstract
Digital humans are emerging as autonomous agents in multiparty interactions, yet existing evaluation metrics largely ignore contextual coordination dynamics. We introduce a unified, intervention-driven framework for objective assessment of multiparty social behaviour in skeletal motion data, spanning three complementary dimensions: (1) synchrony via Cross-Recurrence Quantification Analysis, (2) temporal alignment via Multiscale Empirical Mode Decompositionbased Beat Consistency, and (3) structural similarity via Soft Dynamic Time Warping. We validate metric sensitivity through three theory-driven perturbations -- gesture kinematic dampening, uniform speech-gesture delays, and prosodic pitch-variance reduction-applied to 30-second thin slices of group interactions from the DnD dataset. Mixed-effects analyses reveal predictable, joint-independent shifts: dampening increases…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
