Designing LLM-simulated Immersive Spaces to Enhance Autistic Children's Social Affordances Understanding

TL;DR

This paper presents AIroad, an LLM-based immersive environment designed to improve autistic children's understanding of social cues in traffic, demonstrating significant learning gains and high usability in a user study.

Contribution

The paper introduces a novel LLM-simulated immersive system with 17 design considerations to enhance social affordance understanding in autistic children.

Findings

Significant improvement in social cue comprehension among participants.

High perceived usability reported by parents.

Effective engagement of autistic children in traffic scenarios.

Abstract

One of the key challenges faced by autistic children is understanding social affordances in complex environments, which further impacts their ability to respond appropriately to social signals. In traffic scenarios, this impairment can even lead to safety concerns. In this paper, we introduce an LLM-simulated immersive projection environment designed to improve this ability in autistic children while ensuring their safety. We first propose 17 design considerations across four major categories, derived from a comprehensive review of previous research. Next, we developed a system called AIroad, which leverages LLMs to simulate drivers with varying social intents, expressed through explicit multimodal social signals. AIroad helps autistic children bridge the gap in recognizing the intentions behind behaviors and learning appropriate responses through various stimuli. A user study involving 14 participants demonstrated that this technology effectively engages autistic children and leads to significant improvements in their comprehension of social affordances in traffic scenarios. Additionally, parents reported high perceived usability of the system. These findings highlight the potential of combining LLM technology with immersive environments for the functional rehabilitation of autistic children in the future.