# A hand sign recognition based signal system for mute people using machine learning

## TL;DR

This paper introduces a lightweight system for recognizing hand gestures to help mute individuals communicate more effectively using machine learning.

## Contribution

A novel framework combining MediaPipe hand tracking with classifiers for real-time, low-resource static and dynamic gesture recognition.

## Key findings

- The system achieved 94.1% accuracy with 30 FPS performance on CPU-only hardware.
- It outperformed CNN, Transformer, and TinyML baselines in balancing accuracy and efficiency.
- The method is robust to varied lighting conditions and suitable for assistive communication.

## Abstract

Communication is a keystone of human engagement, yet individuals with speech deficiencies or those operating in perturbations sensitive environments often face pitfalls in conveying their thoughts effectively. Communication among mute people and the general public follows a major limitation, since most people are unknown with sign language and professional communicators are not Continuously attainable. This Limitation commonly Brings about to social discrimination, restricted access to services, and susceptibility on others for regular communication. Hand gesture recognition provides an intuitive channel of communication for mute individuals, but most Prevailing methods are computationally Intensive and unsuitable for real time applies on modest hardware. This study introduces a lightweight framework that aggregates MediaPipe hand landmark detection with supporting information classifiers to recognize both static and dynamic gestures. Seven representative gestures (A, B, C, D, Open, Close, OK) were tested with a balanced dataset of 3500 samples. The system achieved 94.1 % accuracy on a partitioned test set while sustaining 30 FPS in CPU only deployment. Compared with CNN, Transformer, and TinyML baselines, the proposed approach provides a high performing balance of accuracy, efficiency, and accessibility .•Integrates MediaPipe based hand tracking with twofold classifiers for static and dynamic gesture recognition.•Demonstrates real time performance and robustness over varied lighting conditions.•Offers an accessible, low resource method relevant for assistive communication applications.

Integrates MediaPipe based hand tracking with twofold classifiers for static and dynamic gesture recognition.

Demonstrates real time performance and robustness over varied lighting conditions.

Offers an accessible, low resource method relevant for assistive communication applications.

Image, graphical abstract