Non-Verbal Communication Analysis in Victim-Offender Mediations

TL;DR

This paper introduces a non-invasive ambient intelligence system that uses computer vision and social signal processing to analyze non-verbal cues in Victim-Offender Mediations, achieving high accuracy in predicting social responses.

Contribution

It presents a novel framework combining multi-modal data and behavioral indicators for automatic analysis of non-verbal communication in restorative justice settings.

Findings

Achieved 86% accuracy in predicting satisfaction.

Achieved 79% accuracy in predicting agreement and receptivity.

Mean deviation of social signal predictions between 0.5 and 0.7.

Abstract

In this paper we present a non-invasive ambient intelligence framework for the semi-automatic analysis of non-verbal communication applied to the restorative justice field. In particular, we propose the use of computer vision and social signal processing technologies in real scenarios of Victim-Offender Mediations, applying feature extraction techniques to multi-modal audio-RGB-depth data. We compute a set of behavioral indicators that define communicative cues from the fields of psychology and observational methodology. We test our methodology on data captured in real world Victim-Offender Mediation sessions in Catalonia in collaboration with the regional government. We define the ground truth based on expert opinions when annotating the observed social responses. Using different state-of-the-art binary classification approaches, our system achieves recognition accuracies of 86% when predicting satisfaction, and 79% when predicting both agreement and receptivity. Applying a regression strategy, we obtain a mean deviation for the predictions between 0.5 and 0.7 in the range [1-5] for the computed social signals.