Classification in Japanese Sign Language Based on Dynamic Facial Expressions

TL;DR

This paper presents a neural network-based method for recognizing Japanese Sign Language by analyzing facial expressions, achieving over 96% accuracy in classifying sentence types, which enhances communication for deaf individuals.

Contribution

The study introduces a novel JSL recognition approach focusing on facial expressions, addressing the lack of datasets and improving sentence type classification accuracy.

Findings

Achieved 96.05% classification accuracy.

Validated effectiveness of facial expression analysis for JSL recognition.

Highlights importance of non-manual markers in sign language understanding.

Abstract

Sign language is a visual language expressed through hand movements and non-manual markers. Non-manual markers include facial expressions and head movements. These expressions vary across different nations. Therefore, specialized analysis methods for each sign language are necessary. However, research on Japanese Sign Language (JSL) recognition is limited due to a lack of datasets. The development of recognition models that consider both manual and non-manual features of JSL is crucial for precise and smooth communication with deaf individuals. In JSL, sentence types such as affirmative statements and questions are distinguished by facial expressions. In this paper, we propose a JSL recognition method that focuses on facial expressions. Our proposed method utilizes a neural network to analyze facial features and classify sentence types. Through the experiments, we confirm our method's effectiveness by achieving a classification accuracy of 96.05%.