A Comprehensive Study on Deep Learning-based Methods for Sign Language Recognition

TL;DR

This paper evaluates deep learning methods for sign language recognition, introduces new training criteria, discusses pretraining schemes, and presents a novel RGB+D Greek sign language dataset with detailed annotations.

Contribution

It provides a comprehensive comparison of recent deep neural network approaches and introduces the first annotated RGB+D dataset for Greek sign language recognition.

Findings

Deep neural networks outperform traditional methods.

New sequence training criteria improve recognition accuracy.

The Greek sign language dataset enables advanced research.

Abstract

In this paper, a comparative experimental assessment of computer vision-based methods for sign language recognition is conducted. By implementing the most recent deep neural network methods in this field, a thorough evaluation on multiple publicly available datasets is performed. The aim of the present study is to provide insights on sign language recognition, focusing on mapping non-segmented video streams to glosses. For this task, two new sequence training criteria, known from the fields of speech and scene text recognition, are introduced. Furthermore, a plethora of pretraining schemes is thoroughly discussed. Finally, a new RGB+D dataset for the Greek sign language is created. To the best of our knowledge, this is the first sign language dataset where sentence and gloss level annotations are provided for a video capture.