Introducing CALMED: Multimodal Annotated Dataset for Emotion Detection in Children with Autism

TL;DR

This paper presents CALMED, a multimodal annotated dataset of children with autism for emotion detection, aiming to improve personalized HCI systems and facilitate research in affective computing for ASD.

Contribution

The paper introduces CALMED, a new multimodal dataset with audio, video, and annotations for emotion detection in children with autism, sharing it to advance research.

Findings

Created a dataset with 57,012 examples of children with autism.

Collected multimodal data including audio, video, and annotations.

Shared the dataset to support future affective computing research.

Abstract

Automatic Emotion Detection (ED) aims to build systems to identify users' emotions automatically. This field has the potential to enhance HCI, creating an individualised experience for the user. However, ED systems tend to perform poorly on people with Autism Spectrum Disorder (ASD). Hence, the need to create ED systems tailored to how people with autism express emotions. Previous works have created ED systems tailored for children with ASD but did not share the resulting dataset. Sharing annotated datasets is essential to enable the development of more advanced computer models for ED within the research community. In this paper, we describe our experience establishing a process to create a multimodal annotated dataset featuring children with a level 1 diagnosis of autism. In addition, we introduce CALMED (Children, Autism, Multimodal, Emotion, Detection), the resulting multimodal emotion detection dataset featuring children with autism aged 8-12. CALMED includes audio and video features extracted from recording files of study sessions with participants, together with annotations provided by their parents into four target classes. The generated dataset includes a total of 57,012 examples, with each example representing a time window of 200ms (0.2s). Our experience and methods described here, together with the dataset shared, aim to contribute to future research applications of affective computing in ASD, which has the potential to create systems to improve the lives of people with ASD.