iLSU-T: an Open Dataset for Uruguayan Sign Language Translation

TL;DR

This paper introduces iLSU T, a comprehensive open dataset of Uruguayan Sign Language videos with transcriptions, aiming to advance local sign language translation research and tools.

Contribution

It provides the first large-scale, multimodal Uruguayan Sign Language dataset with diverse topics and professional interpreters, enabling new research in sign language translation.

Findings

Baseline results for three translation algorithms are established.

The dataset highlights the need for localized sign language data.

Experiments demonstrate the dataset's usefulness for developing translation tools.

Abstract

Automatic sign language translation has gained particular interest in the computer vision and computational linguistics communities in recent years. Given each sign language country particularities, machine translation requires local data to develop new techniques and adapt existing ones. This work presents iLSU T, an open dataset of interpreted Uruguayan Sign Language RGB videos with audio and text transcriptions. This type of multimodal and curated data is paramount for developing novel approaches to understand or generate tools for sign language processing. iLSU T comprises more than 185 hours of interpreted sign language videos from public TV broadcasting. It covers diverse topics and includes the participation of 18 professional interpreters of sign language. A series of experiments using three state of the art translation algorithms is presented. The aim is to establish a baseline for this dataset and evaluate its usefulness and the proposed pipeline for data processing. The experiments highlight the need for more localized datasets for sign language translation and understanding, which are critical for developing novel tools to improve accessibility and inclusion of all individuals. Our data and code can be accessed.