Enhancing Mathematics Learning for Hard-of-Hearing Students Through Real-Time Palestinian Sign Language Recognition: A New Dataset

TL;DR

This paper presents a new AI-based Palestinian sign language recognition system for mathematics education, utilizing a custom dataset and Vision Transformer, achieving high accuracy to improve accessibility for hard-of-hearing students.

Contribution

The study introduces a novel Palestinian sign language dataset for mathematics and fine-tunes a Vision Transformer model, advancing AI-driven educational tools for inclusivity.

Findings

Model achieved 97.59% accuracy in gesture recognition

Created the first specialized PSL dataset for mathematical gestures

Demonstrated effectiveness of deep learning in accessible education

Abstract

The study aims to enhance mathematics education accessibility for hard-of-hearing students by developing an accurate Palestinian sign language PSL recognition system using advanced artificial intelligence techniques. Due to the scarcity of digital resources for PSL, a custom dataset comprising 41 mathematical gesture classes was created, and recorded by PSL experts to ensure linguistic accuracy and domain specificity. To leverage state-of-the-art-computer vision techniques, a Vision Transformer ViTModel was fine-tuned for gesture classification. The model achieved an accuracy of 97.59%, demonstrating its effectiveness in recognizing mathematical signs with high precision and reliability. This study highlights the role of deep learning in developing intelligent educational tools that bridge the learning gap for hard-of-hearing students by providing AI-driven interactive solutions to enhance mathematical comprehension. This work represents a significant step toward innovative and inclusive frosting digital integration in specialized learning environments. The dataset is hosted on Hugging Face at https://huggingface.co/datasets/fidaakh/STEM_data.