Developing Lightweight DNN Models With Limited Data For Real-Time Sign Language Recognition

TL;DR

This paper introduces a lightweight DNN framework for real-time sign language recognition that performs accurately on limited data and runs efficiently on edge devices, enabling practical deployment.

Contribution

The authors develop a novel, compact DNN architecture and data encoding method tailored for sign language recognition with limited data and computational resources.

Findings

92% accuracy in isolated sign recognition

sub 10ms latency on edge devices

successful integration into a web application

Abstract

We present a novel framework for real-time sign language recognition using lightweight DNNs trained on limited data. Our system addresses key challenges in sign language recognition, including data scarcity, high computational costs, and discrepancies in frame rates between training and inference environments. By encoding sign language specific parameters, such as handshape, palm orientation, movement, and location into vectorized inputs, and leveraging MediaPipe for landmark extraction, we achieve highly separable input data representations. Our DNN architecture, optimized for sub 10MB deployment, enables accurate classification of 343 signs with less than 10ms latency on edge devices. The data annotation platform 'slait data' facilitates structured labeling and vector extraction. Our model achieved 92% accuracy in isolated sign recognition and has been integrated into the 'slait ai' web application, where it demonstrates stable inference.