Leveraging Recurrent Neural Networks for Multimodal Recognition of Social Norm Violation in Dialog

TL;DR

This paper presents a neural network-based approach that uses multimodal data to automatically detect social norm violations in conversations, aiming to enhance social-aware AI agents.

Contribution

It introduces a predictive model leveraging recurrent neural networks and multimodal information to identify social norm violations in discourse, with a focus on temporal and contextual cues.

Findings

Achieved an F1 score of 0.705 in detecting social norm violations.

Demonstrated the effectiveness of multimodal and temporal information in recognition tasks.

Discussed implications for developing socially aware AI agents.

Abstract

Social norms are shared rules that govern and facilitate social interaction. Violating such social norms via teasing and insults may serve to upend power imbalances or, on the contrary reinforce solidarity and rapport in conversation, rapport which is highly situated and context-dependent. In this work, we investigate the task of automatically identifying the phenomena of social norm violation in discourse. Towards this goal, we leverage the power of recurrent neural networks and multimodal information present in the interaction, and propose a predictive model to recognize social norm violation. Using long-term temporal and contextual information, our model achieves an F1 score of 0.705. Implications of our work regarding developing a social-aware agent are discussed.