ADDSL: Hand Gesture Detection and Sign Language Recognition on Annotated Danish Sign Language

TL;DR

This paper presents ADDSL, a new annotated dataset for Danish Sign Language, and trains a YOLOv5-based model achieving 92% accuracy in hand gesture recognition with minimal training images.

Contribution

Introduces the ADDSL dataset and demonstrates a highly accurate, efficient hand gesture recognition model using minimal training data and a modified YOLOv5 architecture.

Findings

Achieved 92% accuracy in sign language recognition

Model inference time is approximately 9ms per image

Effective recognition with only seven images per class

Abstract

For a long time, detecting hand gestures and recognizing them as letters or numbers has been a challenging task. This creates communication barriers for individuals with disabilities. This paper introduces a new dataset, the Annotated Dataset for Danish Sign Language (ADDSL). Annota-tions for the dataset were made using the open-source tool LabelImg in the YOLO format. Using this dataset, a one-stage ob-ject detector model (YOLOv5) was trained with the CSP-DarkNet53 backbone and YOLOv3 head to recognize letters (A-Z) and numbers (0-9) using only seven unique images per class (without augmen-tation). Five models were trained with 350 epochs, resulting in an average inference time of 9.02ms per image and a best accu-racy of 92% when compared to previous research. Our results show that modified model is efficient and more accurate than existing work in the same field. The code repository for our model is available at the GitHub repository https://github.com/s4nyam/pvt-addsl.