Modeling of ASD/TD Children's Behaviors in Interaction with a Virtual Social Robot During a Music Education Program Using Deep Neural Networks

TL;DR

This study developed a deep learning-based system to classify ASD and TD children and generate realistic behaviors during interactions with a virtual social robot in a music education setting.

Contribution

It introduces a novel deep neural network approach for behavior classification and simulation in children with ASD and neurotypical peers during robot interactions.

Findings

Achieved 81% accuracy and 96% sensitivity in distinguishing ASD from TD children.

Designed a transformer-based model that experts struggled to differentiate from real behaviors.

Successfully simulated realistic children's behaviors with high agreement among experts.

Abstract

This research aimed to develop an intelligent system to evaluate performance and extract behavioral models for children with ASD and neurotypical (TD) children by interacting with a virtual social robot in a music education program using deep neural networks. The system has two main features: 1) it distinguishes between neurotypical children and those with ASD based on their behavior, and 2) generates behaviors resembling those of neurotypical or ASD children in similar situations using deep learning. Intelligent systems that identify complex patterns and simulate behavior can aid in diagnosis, therapist training, and understanding the disorder. Using data from a previous study at the Social and Cognitive Robotics Laboratory of Sharif University of Technology (including the usable data of 9 ASD and 21 TD participants), the system achieved an accuracy of 81% and sensitivity of 96% in distinguishing neurotypical children from those with ASD using both impact data and motion signals. A transformer-based network was designed to reproduce children's behaviors. Experts in the field struggled to differentiate real behaviors from reproduced ones, with an accuracy of 53.5% and agreement of 68%, indicating the model's success in simulating realistic behaviors.