DAiSEE: Towards User Engagement Recognition in the Wild

TL;DR

DAiSEE is a pioneering multi-label video dataset capturing user affective states like boredom and frustration in real-world settings, enabling research in affect recognition and machine learning methods.

Contribution

It introduces the first large-scale, multi-label video dataset with expert annotations for user affective states in natural environments.

Findings

Benchmark results established using state-of-the-art video classification methods.

The dataset facilitates research in feature extraction and context-based inference.

Provides a foundation for developing machine learning models for affect recognition.

Abstract

We introduce DAiSEE, the first multi-label video classification dataset comprising of 9068 video snippets captured from 112 users for recognizing the user affective states of boredom, confusion, engagement, and frustration in the wild. The dataset has four levels of labels namely - very low, low, high, and very high for each of the affective states, which are crowd annotated and correlated with a gold standard annotation created using a team of expert psychologists. We have also established benchmark results on this dataset using state-of-the-art video classification methods that are available today. We believe that DAiSEE will provide the research community with challenges in feature extraction, context-based inference, and development of suitable machine learning methods for related tasks, thus providing a springboard for further research. The dataset is available for download at https://people.iith.ac.in/vineethnb/resources/daisee/index.html.