Voice-Controlled Scratch for Children with (Motor) Disabilities

TL;DR

This paper presents MeowCrophone, a voice control system for Scratch that significantly improves accessibility for children with motor disabilities by enabling code editing through robust voice commands.

Contribution

It introduces a multi-stage voice recognition pipeline that enhances command accuracy, making Scratch more accessible to children with motor disabilities.

Findings

Baseline speech recognition had 46.4% success rate.

MeowCrophone improved accuracy to 82.8%.

Simple commands reached 96.9% accuracy.

Abstract

Block-based programming environments like Scratch have become widely adopted in Computer Science Education, but the mouse-based drag-and-drop interface can challenge users with disabilities. While prior work has provided solutions supporting children with visual impairment, these solutions tend to focus on making content perceivable and do not address the physical interaction barriers faced by users with motor disabilities. To bridge this gap, we introduce MeowCrophone, an approach that uses voice control to allow editing code in Scratch. MeowCrophone supports clicking elements, placing blocks, and navigating the workspace via a multi-modal voice user interface that uses numerical overlays and label reading to bypass physical input entirely. As imperfect speech recognition is common in classrooms and for children with dysarthria, MeowCrophone employs a multi-stage matching pipeline using regular expressions, phonetic matching, and a custom grammar. Evaluation shows that while free speech recognition systems achieved a baseline success rate of only 46.4%, MeowCrophone's pipeline improved results to 82.8% overall, with simple commands reaching 96.9% accuracy. This demonstrates that robust voice control can make Scratch accessible to users for whom visual aids are insufficient.