BAUST Lipi: A BdSL Dataset with Deep Learning Based Bangla Sign Language Recognition

TL;DR

This paper introduces a comprehensive Bangla Sign Language dataset with 18,000 images and a hybrid CNN-LSTM model that achieves nearly 98% accuracy, advancing BdSL recognition research.

Contribution

The work provides the first large-scale BdSL dataset and a novel hybrid deep learning model for improved sign language recognition accuracy.

Findings

Achieved 97.92% accuracy on BdSL recognition

Created a dataset of 18,000 images of 36 Bengali symbols

Demonstrated the effectiveness of a hybrid CNN-LSTM model

Abstract

People commonly communicate in English, Arabic, and Bengali spoken languages through various mediums. However, deaf and hard-of-hearing individuals primarily use body language and sign language to express their needs and achieve independence. Sign language research is burgeoning to enhance communication with the deaf community. While many researchers have made strides in recognizing sign languages such as French, British, Arabic, Turkish, and American, there has been limited research on Bangla sign language (BdSL) with less-than-satisfactory results. One significant barrier has been the lack of a comprehensive Bangla sign language dataset. In our work, we introduced a new BdSL dataset comprising alphabets totaling 18,000 images, with each image being 224x224 pixels in size. Our dataset encompasses 36 Bengali symbols, of which 30 are consonants and the remaining six are vowels. Despite our dataset contribution, many existing systems continue to grapple with achieving high-performance accuracy for BdSL. To address this, we devised a hybrid Convolutional Neural Network (CNN) model, integrating multiple convolutional layers, activation functions, dropout techniques, and LSTM layers. Upon evaluating our hybrid-CNN model with the newly created BdSL dataset, we achieved an accuracy rate of 97.92\%. We are confident that both our BdSL dataset and hybrid CNN model will be recognized as significant milestones in BdSL research.