SDW-ASL: A Dynamic System to Generate Large Scale Dataset for Continuous American Sign Language

TL;DR

This paper introduces SDW-ASL, a dynamic system that generates large-scale, continuous American Sign Language datasets to facilitate deep learning research in sign language processing.

Contribution

The authors developed a scalable system for creating extensive ASL datasets, including the largest continuous sign language dataset to date, supporting improved sign language recognition and production.

Findings

Released a 104-hour continuous ASL dataset with 30k sentences

System can evolve to incorporate better data processing techniques

Facilitates deep learning research in ASL natural language processing

Abstract

Despite tremendous progress in natural language processing using deep learning techniques in recent years, sign language production and comprehension has advanced very little. One critical barrier is the lack of largescale datasets available to the public due to the unbearable cost of labeled data generation. Efforts to provide public data for American Sign Language (ASL) comprehension have yielded two datasets, comprising more than thousand video clips. These datasets are large enough to enable a meaningful start to deep learning research on sign languages but are far too small to lead to any solution that can be practically deployed. So far, there is still no suitable dataset for ASL production. We proposed a system that can generate large scale ASL datasets for continuous ASL. It is suitable for general ASL processing and is particularly useful for ASL production. The continuous ASL dataset contains English labeled human articulations in condensed body pose data formats. To better serve the research community, we are releasing the first version of our ASL dataset, which contains 30k sentences, 416k words, a vocabulary of 18k words, in a total of 104 hours. This is the largest continuous sign language dataset published to date in terms of video duration. We also describe a system that can evolve and expand the dataset to incorporate better data processing techniques and more contents when available. It is our hope that the release of this ASL dataset and the sustainable dataset generation system to the public will propel better deep-learning research in ASL natural language processing.