On the Importance of Sign Labeling: The Hamburg Sign Language Notation System Case Study

TL;DR

This paper analyzes the effectiveness and consistency of the Hamburg Sign Language Notation System (HamNoSys) labels across different sign languages, highlighting challenges and limitations in current manual labeling practices for machine learning applications.

Contribution

It provides a comprehensive analysis of HamNoSys label consistency across multiple sign languages, identifying key challenges and limitations in current manual labeling methods.

Findings

HamNoSys labels show variability across different maintainers and sign languages.

Manual labeling remains a significant bottleneck for scalable sign language recognition.

The study highlights the need for automated and standardized labeling solutions.

Abstract

Labeling is the cornerstone of supervised machine learning, which has been exploited in a plethora of various applications, with sign language recognition being one of them. However, such algorithms must be fed with a huge amount of consistently labeled data during the training process to elaborate a well-generalizing model. In addition, there is a great need for an automated solution that works with any nationally diversified sign language. Although there are language-agnostic transcription systems, such as the Hamburg Sign Language Notation System (HamNoSys) that describe the signer's initial position and body movement instead of the glosses' meanings, there are still issues with providing accurate and reliable labels for every real-world use case. In this context, the industry relies heavily on manual attribution and labeling of the available video data. In this work, we tackle this issue and thoroughly analyze the HamNoSys labels provided by various maintainers of open sign language corpora in five sign languages, in order to examine the challenges encountered in labeling video data. We also investigate the consistency and objectivity of HamNoSys-based labels for the purpose of training machine learning models. Our findings provide valuable insights into the limitations of the current labeling methods and pave the way for future research on developing more accurate and efficient solutions for sign language recognition.