A Deep Learning Approach to Tongue Detection for Pediatric Population

TL;DR

This study develops and evaluates a CNN-based tongue gesture recognition system for children, demonstrating high accuracy in naturalistic settings and emphasizing the importance of pediatric-specific training data.

Contribution

The paper introduces a novel CNN architecture for tongue gesture detection in children and highlights the effectiveness of pediatric data for training.

Findings

Achieved up to 99% accuracy in classifying tongue-out gestures.

Using children data for training improves performance over adult data.

Validated the approach in naturalistic play settings.

Abstract

Children with severe disabilities and complex communication needs face limitations in the usage of access technology (AT) devices. Conventional ATs (e.g., mechanical switches) can be insufficient for nonverbal children and those with limited voluntary motion control. Automatic techniques for the detection of tongue gestures represent a promising pathway. Previous studies have shown the robustness of tongue detection algorithms on adult participants, but further research is needed to use these methods with children. In this study, a network architecture for tongue-out gesture recognition was implemented and evaluated on videos recorded in a naturalistic setting when children were playing a video-game. A cascade object detector algorithm was used to detect the participants' faces, and an automated classification scheme for tongue gesture detection was developed using a convolutional neural network (CNN). In evaluation experiments conducted, the network was trained using adults and children's images. The network classification accuracy was evaluated using leave-one-subject-out cross-validation. Preliminary classification results obtained from the analysis of videos of five typically developing children showed an accuracy of up to 99% in predicting tongue-out gestures. Moreover, we demonstrated that using only children data for training the classifier yielded better performance than adult's one supporting the need for pediatric tongue gesture datasets.