Comparison of Autoencoders for tokenization of ASL datasets

TL;DR

This paper compares different autoencoder architectures for tokenizing ASL datasets, finding that diffusion autoencoders provide superior image reconstruction for sign language applications.

Contribution

It introduces and evaluates three autoencoder types for ASL image tokenization, highlighting the effectiveness of diffusion autoencoders over traditional methods.

Findings

Diffusion autoencoders achieved lowest MSE and highest MOS.

Convolutional autoencoders effectively extracted spatial features.

Feedforward autoencoders showed limitations with complex images.

Abstract

Generative AI, powered by large language models (LLMs), has revolutionized applications across text, audio, images, and video. This study focuses on developing and evaluating encoder-decoder architectures for the American Sign Language (ASL) image dataset, consisting of 87,000 images across 29 hand sign classes. Three approaches were compared: Feedforward Autoencoders, Convolutional Autoencoders, and Diffusion Autoencoders. The Diffusion Autoencoder outperformed the others, achieving the lowest mean squared error (MSE) and highest Mean Opinion Score (MOS) due to its probabilistic noise modeling and iterative denoising capabilities. The Convolutional Autoencoder demonstrated effective spatial feature extraction but lacked the robustness of the diffusion process, while the Feedforward Autoencoder served as a baseline with limitations in handling complex image data. Objective and subjective evaluations confirmed the superiority of the Diffusion Autoencoder for high-fidelity image reconstruction, emphasizing its potential in multimodal AI applications such as sign language recognition and generation. This work provides critical insights into designing robust encoder-decoder systems to advance multimodal AI capabilities.