Proper Body Landmark Subset Enables More Accurate and 5X Faster Recognition of Isolated Signs in LIBRAS

TL;DR

This study introduces a lightweight body landmark subset for LIBRAS sign recognition, achieving over 5X faster processing and comparable or better accuracy than existing methods, with effective missing landmark mitigation.

Contribution

The paper proposes a novel landmark subset selection strategy and spline-based imputation to enhance recognition speed and accuracy in LIBRAS sign language.

Findings

Proper landmark subset matches or exceeds state-of-the-art accuracy.

Processing time is reduced by more than 5X with the proposed method.

Spline-based imputation improves landmark detection robustness.

Abstract

This paper examines the feasibility of utilizing lightweight body landmark detection for recognizing isolated signs in Brazilian Sign Language (LIBRAS). Although the use of skeleton-image representation has enabled substantial improvements in recognition performance, the use of OpenPose for landmark extraction hindered time performance. In a preliminary investigation, we observed that simply replacing OpenPose with lightweight MediaPipe, while improving processing speed, significantly reduced accuracy. To overcome this limitation, we explored landmark subset selection strategies to optimize recognition performance. Experimental results show that a proper landmark subset achieves comparable or superior performance to state-of-the-art methods while reducing processing time by more than 5X. As an additional contribution, we demonstrate that spline-based imputation effectively mitigates missing landmark issues, leading to substantial accuracy gains.