GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos

TL;DR

GAViD introduces a large-scale multimodal dataset with annotations for group affect, enabling improved context-aware recognition of emotions in social videos, and presents a neural network model achieving competitive accuracy.

Contribution

The paper provides the GAViD dataset with multimodal, annotated social videos and proposes CAGNet, a model for context-aware group affect recognition, advancing research in social affect analysis.

Findings

CAGNet achieves 63.20% accuracy on GAViD.

GAViD dataset contains 5091 annotated video clips.

Code and dataset are publicly available at github.com/deepakkumar-iitr/GAViD.

Abstract

Understanding affective dynamics in real-world social systems is fundamental to modeling and analyzing human-human interactions in complex environments. Group affect emerges from intertwined human-human interactions, contextual influences, and behavioral cues, making its quantitative modeling a challenging computational social systems problem. However, computational modeling of group affect in in-the-wild scenarios remains challenging due to limited large-scale annotated datasets and the inherent complexity of multimodal social interactions shaped by contextual and behavioral variability. The lack of comprehensive datasets annotated with multimodal and contextual information further limits advances in the field. To address this, we introduce the Group Affect from ViDeos (GAViD) dataset, comprising 5091 video clips with multimodal data (video, audio and context), annotated with ternary valence and discrete emotion labels and enriched with VideoGPT-generated contextual metadata and human-annotated action cues. We also present Context-Aware Group Affect Recognition Network (CAGNet) for multimodal context-aware group affect recognition. CAGNet achieves 63.20\% test accuracy on GAViD, comparable to state-of-the-art performance. The dataset and code are available at github.com/deepakkumar-iitr/GAViD.