A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset

TL;DR

This paper introduces a new Greek Sign Language dataset based on elementary school curriculum, enabling more realistic and extensive end-to-end sign language translation research.

Contribution

The paper presents a large, diverse Greek Sign Language dataset for end-to-end translation, filling a critical resource gap in SLT research.

Findings

The dataset contains 29,653 video-translation pairs.

State-of-the-art Transformer models trained on the dataset show promising results.

The dataset covers a wide range of elementary school subjects.

Abstract

Automatic Sign Language Translation (SLT) is a research avenue of great societal impact. End-to-End SLT facilitates the interaction of Hard-of-Hearing (HoH) with hearing people, thus improving their social life and opportunities for participation in social life. However, research within this frame of reference is still in its infancy, and current resources are particularly limited. Existing SLT methods are either of low translation ability or are trained and evaluated on datasets of restricted vocabulary and questionable real-world value. A characteristic example is Phoenix2014T benchmark dataset, which only covers weather forecasts in German Sign Language. To address this shortage of resources, we introduce a newly constructed collection of 29653 Greek Sign Language video-translation pairs which is based on the official syllabus of Greek Elementary School. Our dataset covers a wide range of subjects. We use this novel dataset to train recent state-of-the-art Transformer-based methods widely used in SLT research. Our results demonstrate the potential of our introduced dataset to advance SLT research by offering a favourable balance between usability and real-world value.