Visual Methods for Sign Language Recognition: A Modality-Based Review

TL;DR

This review paper analyzes recent visual recognition methods for sign language, focusing on unimodal and multimodal approaches, datasets, and challenges, aiming to inform future development of interactive sign language services.

Contribution

It provides a comprehensive organization and comparison of sign language visual recognition methods based on input modalities and pipeline steps, highlighting open research paths.

Findings

Analysis of datasets and approaches for sign language recognition

Comparison of unimodal and multimodal methods

Identification of open challenges and future directions

Abstract

Sign language visual recognition from continuous multi-modal streams is still one of the most challenging fields. Recent advances in human actions recognition are exploiting the ascension of GPU-based learning from massive data, and are getting closer to human-like performances. They are then prone to creating interactive services for the deaf and hearing-impaired communities. A population that is expected to grow considerably in the years to come. This paper aims at reviewing the human actions recognition literature with the sign-language visual understanding as a scope. The methods analyzed will be mainly organized according to the different types of unimodal inputs exploited, their relative multi-modal combinations and pipeline steps. In each section, we will detail and compare the related datasets, approaches then distinguish the still open contribution paths suitable for the creation of sign language related services. Special attention will be paid to the approaches and commercial solutions handling facial expressions and continuous signing.