Grammatical facial expression recognition using customized deep neural network architecture

TL;DR

This paper introduces a customized deep neural network architecture for recognizing facial expressions associated with Brazilian sign language, significantly improving understanding of emotional context in sign language communication.

Contribution

It presents a novel deep neural network architecture tailored for facial expression recognition in sign language, enhancing emotional understanding beyond traditional sign classification.

Findings

Achieved 98.04% overall accuracy in facial expression recognition.

Demonstrated improved pattern learning with the customized architecture.

Validated effectiveness for practical sign language interpretation scenarios.

Abstract

This paper proposes to expand the visual understanding capacity of computers by helping it recognize human sign language more efficiently. This is carried out through recognition of facial expressions, which accompany the hand signs used in this language. This paper specially focuses on the popular Brazilian sign language (LIBRAS). While classifying different hand signs into their respective word meanings has already seen much literature dedicated to it, the emotions or intention with which the words are expressed haven't primarily been taken into consideration. As from our normal human experience, words expressed with different emotions or mood can have completely different meanings attached to it. Lending computers the ability of classifying these facial expressions, can help add another level of deep understanding of what the deaf person exactly wants to communicate. The proposed idea is implemented through a deep neural network having a customized architecture. This helps learning specific patterns in individual expressions much better as compared to a generic approach. With an overall accuracy of 98.04%, the implemented deep network performs excellently well and thus is fit to be used in any given practical scenario.