Automatic Hand Sign Recognition: Identify Unusuality through Latent Cognizance

TL;DR

This paper introduces new methods for detecting non-sign postures in sign language recognition, addressing limitations of confidence ratio, and provides insights into the underlying inference mechanisms, improving accuracy in Thai Finger Spelling recognition.

Contribution

It proposes alternative formulations to confidence ratio for non-sign detection and explores their effectiveness and interpretability in sign language transcription.

Findings

Proposed methods improve non-sign detection accuracy.

Alternative formulations address computational instability.

Insights reveal hidden meanings in learned inference models.

Abstract

Sign language is a main communication channel among hearing disability community. Automatic sign language transcription could facilitate better communication and understanding between hearing disability community and hearing majority. As a recent work in automatic sign language transcription has discussed, effectively handling or identifying a non-sign posture is one of the key issues. A non-sign posture is a posture unintended for sign reading and does not belong to any valid sign. A non-sign posture may arise during sign transition or simply from an unaware posture. Confidence ratio has been proposed to mitigate the issue. Confidence ratio is simple to compute and readily available without extra training. However, confidence ratio is reported to only partially address the problem. In addition, confidence ratio formulation is susceptible to computational instability. This article proposes alternative formulations to confidence ratio, investigates an issue of non-sign identification for Thai Finger Spelling recognition, explores potential solutions and has found a promising direction. Not only does this finding address the issue of non-sign identification, it also provide some insight behind a well-learned inference machine, revealing hidden meaning and new interpretation of the underlying mechanism. Our proposed methods are evaluated and shown to be effective for non-sign detection.