Generation of Indian Sign Language Letters, Numbers, and Words

TL;DR

This paper introduces a new GAN-based model that generates high-quality Indian Sign Language images and provides a large dataset, improving image quality metrics over traditional models.

Contribution

A novel GAN variant combining ProGAN and SAGAN for high-resolution, feature-rich sign language image generation, along with a comprehensive dataset.

Findings

Outperforms ProGAN in Inception Score and FID metrics

Generates high-resolution, feature-rich sign language images

Provides a large dataset of sign language images

Abstract

Sign language, which contains hand movements, facial expressions and bodily gestures, is a significant medium for communicating with hard-of-hearing people. A well-trained sign language community communicates easily, but those who don't know sign language face significant challenges. Recognition and generation are basic communication methods between hearing and hard-of-hearing individuals. Despite progress in recognition, sign language generation still needs to be explored. The Progressive Growing of Generative Adversarial Network (ProGAN) excels at producing high-quality images, while the Self-Attention Generative Adversarial Network (SAGAN) generates feature-rich images at medium resolutions. Balancing resolution and detail is crucial for sign language image generation. We are developing a Generative Adversarial Network (GAN) variant that combines both models to generate feature-rich, high-resolution, and class-conditional sign language images. Our modified Attention-based model generates high-quality images of Indian Sign Language letters, numbers, and words, outperforming the traditional ProGAN in Inception Score (IS) and Fr\'echet Inception Distance (FID), with improvements of 3.2 and 30.12, respectively. Additionally, we are publishing a large dataset incorporating high-quality images of Indian Sign Language alphabets, numbers, and 129 words.