Attention vs LSTM: Improving Word-level BISINDO Recognition

TL;DR

This paper compares LSTM and 1D CNN + Transformer models for Indonesian sign language recognition, demonstrating that while LSTM has lower latency, the 1DCNNTrans model offers higher accuracy and stability for complex gestures.

Contribution

It introduces a comparative analysis of LSTM and 1DCNNTrans models for sign language recognition, highlighting the strengths and weaknesses of each in a practical application.

Findings

LSTM achieved 94.67% accuracy with lower inference latency.

1DCNNTrans achieved 96.12% accuracy and better stability for complex classes.

Both models exceeded 90% validation accuracy for sign language gestures.

Abstract

Indonesia ranks fourth globally in the number of deaf cases. Individuals with hearing impairments often find communication challenging, necessitating the use of sign language. However, there are limited public services that offer such inclusivity. On the other hand, advancements in artificial intelligence (AI) present promising solutions to overcome communication barriers faced by the deaf. This study aims to explore the application of AI in developing models for a simplified sign language translation app and dictionary, designed for integration into public service facilities, to facilitate communication for individuals with hearing impairments, thereby enhancing inclusivity in public services. The researchers compared the performance of LSTM and 1D CNN + Transformer (1DCNNTrans) models for sign language recognition. Through rigorous testing and validation, it was found that the LSTM model achieved an accuracy of 94.67%, while the 1DCNNTrans model achieved an accuracy of 96.12%. Model performance evaluation indicated that although the LSTM exhibited lower inference latency, it showed weaknesses in classifying classes with similar keypoints. In contrast, the 1DCNNTrans model demonstrated greater stability and higher F1 scores for classes with varying levels of complexity compared to the LSTM model. Both models showed excellent performance, exceeding 90% validation accuracy and demonstrating rapid classification of 50 sign language gestures.