AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work

TL;DR

AVCAffe is a comprehensive, large-scale audio-visual dataset capturing cognitive load and affective states during simulated remote work, enabling advanced research in affective computing and remote collaboration.

Contribution

This paper introduces AVCAffe, the first large-scale, original audio-visual dataset of cognitive load and affect in remote work scenarios, with extensive annotations and diverse participant demographics.

Findings

Largest affective dataset in English with 58,000+ clips

Provides ground truth labels for arousal, valence, and cognitive load

Facilitates research on affect classification and remote work impact

Abstract

We introduce AVCAffe, the first Audio-Visual dataset consisting of Cognitive load and Affect attributes. We record AVCAffe by simulating remote work scenarios over a video-conferencing platform, where subjects collaborate to complete a number of cognitively engaging tasks. AVCAffe is the largest originally collected (not collected from the Internet) affective dataset in English language. We recruit 106 participants from 18 different countries of origin, spanning an age range of 18 to 57 years old, with a balanced male-female ratio. AVCAffe comprises a total of 108 hours of video, equivalent to more than 58,000 clips along with task-based self-reported ground truth labels for arousal, valence, and cognitive load attributes such as mental demand, temporal demand, effort, and a few others. We believe AVCAffe would be a challenging benchmark for the deep learning research community given the inherent difficulty of classifying affect and cognitive load in particular. Moreover, our dataset fills an existing timely gap by facilitating the creation of learning systems for better self-management of remote work meetings, and further study of hypotheses regarding the impact of remote work on cognitive load and affective states.