BdSpell: A YOLO-based Real-time Finger Spelling System for Bangla Sign Language

TL;DR

This paper introduces BdSpell, a real-time Bangla Sign Language finger spelling system based on YOLOv5, achieving high accuracy and speed, and improving accessibility for the Bangla Sign Language community.

Contribution

The paper presents a novel YOLOv5-based system that efficiently generates hidden and compound characters without extra classes, enhancing real-time Bangla Sign Language interpretation.

Findings

Character spelling in 1.32 seconds with 98% accuracy

YOLOv5 model trained on 9147 images with 96.4% mAP

Significantly improves BdSL interpretation and accessibility

Abstract

In the domain of Bangla Sign Language (BdSL) interpretation, prior approaches often imposed a burden on users, requiring them to spell words without hidden characters, which were subsequently corrected using Bangla grammar rules due to the missing classes in BdSL36 dataset. However, this method posed a challenge in accurately guessing the incorrect spelling of words. To address this limitation, we propose a novel real-time finger spelling system based on the YOLOv5 architecture. Our system employs specified rules and numerical classes as triggers to efficiently generate hidden and compound characters, eliminating the necessity for additional classes and significantly enhancing user convenience. Notably, our approach achieves character spelling in an impressive 1.32 seconds with a remarkable accuracy rate of 98\%. Furthermore, our YOLOv5 model, trained on 9147 images, demonstrates an exceptional mean Average Precision (mAP) of 96.4\%. These advancements represent a substantial progression in augmenting BdSL interpretation, promising increased inclusivity and accessibility for the linguistic minority. This innovative framework, characterized by compatibility with existing YOLO versions, stands as a transformative milestone in enhancing communication modalities and linguistic equity within the Bangla Sign Language community.