The state of play of ASC-Inclusion: An Integrated Internet-Based Environment for Social Inclusion of Children with Autism Spectrum Conditions

TL;DR

This paper reviews the development of ASC-Inclusion, an internet-based platform designed to improve social communication skills in children with Autism Spectrum Conditions through multimodal interaction and serious gaming.

Contribution

It introduces an integrated virtual environment that combines gesture, facial, and vocal analysis for social skill training in children with ASC.

Findings

Successful integration of multimodal analysis techniques

Positive preliminary results in user engagement and emotion recognition

Framework for real-time feedback in social skill development

Abstract

Individuals with Autism Spectrum Conditions (ASC) have marked difficulties using verbal and non-verbal communication for social interaction. The running ASC-Inclusion project aims to help children with ASC by allowing them to learn how emotions can be expressed and recognised via playing games in a virtual world. The platform includes analysis of users' gestures, facial, and vocal expressions using standard microphone and web-cam or a depth sensor, training through games, text communication with peers, animation, video and audio clips. We present the state of play in realising such a serious game platform and provide results for the different modalities.