Reconnaissance Automatique des Langues des Signes : Une Approche Hybrid\'ee CNN-LSTM Bas\'ee sur Mediapipe

TL;DR

This paper presents a real-time automatic sign language recognition system using a hybrid CNN-LSTM model and Mediapipe, achieving 92% accuracy and aiding communication for deaf communities.

Contribution

It introduces a novel hybrid CNN-LSTM architecture combined with Mediapipe for gesture keypoint extraction, enabling real-time sign language translation.

Findings

Achieved 92% average accuracy on sign language gestures.

Performed well on distinct gestures like 'Hello' and 'Thank you'.

Some confusion remains for visually similar gestures such as 'Call' and 'Yes'.

Abstract

Sign languages play a crucial role in the communication of deaf communities, but they are often marginalized, limiting access to essential services such as healthcare and education. This study proposes an automatic sign language recognition system based on a hybrid CNN-LSTM architecture, using Mediapipe for gesture keypoint extraction. Developed with Python, TensorFlow and Streamlit, the system provides real-time gesture translation. The results show an average accuracy of 92\%, with very good performance for distinct gestures such as ``Hello'' and ``Thank you''. However, some confusions remain for visually similar gestures, such as ``Call'' and ``Yes''. This work opens up interesting perspectives for applications in various fields such as healthcare, education and public services.