MeloKids: Multisensory VR System to Enhance Speech and Motor Coordination in Children with Hearing Loss

TL;DR

This paper investigates a multisensory VR system designed to improve speech and motor skills in children with hearing loss, using fNIRS to assess brain activation during interactive tasks.

Contribution

It introduces a novel multisensory VR rehabilitation system and evaluates its effectiveness through cortical activation patterns in children with hearing impairments.

Findings

Enhanced cortical activation during multisensory VR tasks

Potential for personalized rehabilitation programs

Improved engagement and motor coordination

Abstract

Children with hearing impairments face ongoing challenges in language and motor development. This study explores how multi-sensory feedback technology based on virtual reality (VR), integrating auditory, visual, and tactile stimuli, can enhance rehabilitation outcomes. Using functional near-infrared spectroscopy (fNIRS) technology, we assessed cortical activation patterns in children during pitch-matching tasks across different interaction modes. Our findings aim to provide evidence for designing personalized, interactive rehabilitation systems that enhance cognitive engagement and motor control in children with hearing impairments.