IsharaKotha: A Comprehensive Avatar-based Bangla Sign Language Corpus

TL;DR

IsharaKotha introduces a comprehensive Bangla Sign Language corpus and a deep learning-based translation system, facilitating sign language learning and communication for the hearing-impaired community.

Contribution

It is the first HamNoSys-based Bangla Sign Language corpus combined with a sign translation system using deep learning techniques.

Findings

Achieved an average sign animation quality rating of 3.14 out of 4.

Demonstrated the system's potential for effective sign language translation.

Provided an evaluation interface for sign animation assessment.

Abstract

Sign language is a vital communication medium for the hearing-impaired community, enabling effective interaction and self-expression. To help bridge the communication gap between hearing and hearing-impaired individuals, a text-to-sign translation system is essential. Such systems can also support learners interested in acquiring sign language skills. This work presents IsharaKotha, the first HamNoSys-based Bangla Sign Language corpus, containing 3823 words. A deep learning based lemmatizer was integrated to extract root words, enabling sign generation for complete sentences. An evaluation interface was developed to assess the quality of sign animations for letters, digits, and sentences. Two professional interpreters and one real sign language user rated the animations using categorical numeric scores. The system achieved an average rating of 3.14 out of 4.00, indicating high quality performance between Good and Excellent. These results demonstrate the potential of IsharaKotha to support future advancements in dynamic sign language translation systems. The evaluation system is available at http://bdsl-isharakotha.ap-1.evennode.com