Design and Implementation of a Virtual 3D Educational Environment to improve Deaf Education

TL;DR

This paper presents a virtual 3D educational environment utilizing VR and X3D standards to automatically generate sign language animations from Arabic text, aiming to improve Deaf education through innovative translation tools.

Contribution

It introduces a novel system for Arabic to Algerian Sign Language translation using virtual reality, with automatic sign generation and evaluation by native Deaf users.

Findings

Successful automatic sign language sentence generation

Realistic and smooth gestural sequences achieved

Positive evaluation by Algerian Deaf participants

Abstract

Advances in NLP, knowledge representation and computer graphic technologies can provide us insights into the development of educational tool for Deaf people. Actual education materials and tools for deaf pupils present several problems, since textbooks are designed to support normal students in the classroom and most of them are not suitable for people with hearing disabilities. Virtual Reality (VR) technologies appear to be a good tool and a promising framework in the education of pupils with hearing disabilities. In this paper, we present a current research tasks surrounding the design and implementation of a virtual 3D educational environment based on X3D and H-Anim standards. The system generates and animates automatically Sign language sentence from a semantic representation that encode the whole meaning of the Arabic input text. Some aspects and issues in Sign language generation will be discussed, including the model of Sign representation that facilitate reuse and reduces the time of Sign generation, conversion of semantic components to sign features representation with regard to Sign language linguistics characteristics and how to generate realistic smooth gestural sequences using X3D content to performs transition between signs for natural-looking of animated avatar. Sign language sentences were evaluated by Algerian native Deaf people. The goal of the project is the development of a machine translation system from Arabic to Algerian Sign Language that can be used as educational tool for Deaf children in algerian primary schools.