GLaM-Sign: Greek Language Multimodal Lip Reading with Integrated Sign Language Accessibility

TL;DR

GLaM-Sign is a multimodal Greek language dataset combining audio, video, text, and sign language to improve accessibility for DHH individuals, with applications in translation and subtitle synchronization.

Contribution

It introduces a comprehensive multimodal dataset for Greek Sign Language, advancing inclusive AI and real-time translation technologies.

Findings

Supports real-time sign language translation

Enhances subtitle synchronization for Greek content

Lays groundwork for scalable multilingual accessibility

Abstract

The Greek Language Multimodal Lip Reading with Integrated Sign Language Accessibility (GLaM-Sign) [1] is a groundbreaking resource in accessibility and multimodal AI, designed to support Deaf and Hard-of-Hearing (DHH) individuals. Developed from the FEELIT project [2], it integrates high-resolution audio, video, textual transcriptions, and Greek Sign Language translations for applications like real-time sign language translation and enhanced subtitle synchronization. While its primary focus is on promoting inclusivity in the Greek tourism sector, its adaptability extends to education, healthcare, and public services. Future advancements will enhance word-level precision and scalability to additional languages, supported by advanced AI methodologies and collaborations with diverse stakeholders. This dataset underscores the transformative potential of multimodal resources in bridging communication gaps, fostering innovation, and setting a benchmark for ethical AI and inclusive technologies.