ISLTranslate: Dataset for Translating Indian Sign Language

TL;DR

This paper introduces ISLTranslate, the largest dataset for continuous Indian Sign Language translation, enabling improved development of translation systems and providing a benchmark for future research.

Contribution

It presents the first large-scale ISL-English translation dataset and benchmarks existing translation models on this resource.

Findings

ISLTranslate contains 31,000 sentence pairs.

Benchmark results show current models' performance on ISL translation.

The dataset facilitates future research in Indian Sign Language translation.

Abstract

Sign languages are the primary means of communication for many hard-of-hearing people worldwide. Recently, to bridge the communication gap between the hard-of-hearing community and the rest of the population, several sign language translation datasets have been proposed to enable the development of statistical sign language translation systems. However, there is a dearth of sign language resources for the Indian sign language. This resource paper introduces ISLTranslate, a translation dataset for continuous Indian Sign Language (ISL) consisting of 31k ISL-English sentence/phrase pairs. To the best of our knowledge, it is the largest translation dataset for continuous Indian Sign Language. We provide a detailed analysis of the dataset. To validate the performance of existing end-to-end Sign language to spoken language translation systems, we benchmark the created dataset with a transformer-based model for ISL translation.