HandCraft: Dynamic Sign Generation for Synthetic Data Augmentation
Gaston Gustavo Rios, Pedro Dal Bianco, Franco Ronchetti, Facundo Quiroga, Oscar Stanchi, Santiago Ponte Ah\'on, Waldo Hasperu\'e

TL;DR
This paper introduces a lightweight sign generation model and synthetic data pretraining approach that significantly improve Sign Language Recognition accuracy, outperforming traditional methods and establishing new state-of-the-art results across multiple datasets.
Contribution
The paper presents a novel, computationally efficient sign generation model based on CMLPe and a synthetic data pretraining method that enhances SLR performance.
Findings
Synthetic pretraining outperforms traditional augmentation in some cases.
Combining synthetic data with augmentation yields complementary benefits.
Achieves state-of-the-art results on LSFB and DiSPLaY datasets.
Abstract
Sign Language Recognition (SLR) models face significant performance limitations due to insufficient training data availability. In this article, we address the challenge of limited data in SLR by introducing a novel and lightweight sign generation model based on CMLPe. This model, coupled with a synthetic data pretraining approach, consistently improves recognition accuracy, establishing new state-of-the-art results for the LSFB and DiSPLaY datasets using our Mamba-SL and Transformer-SL classifiers. Our findings reveal that synthetic data pretraining outperforms traditional augmentation methods in some cases and yields complementary benefits when implemented alongside them. Our approach democratizes sign generation and synthetic data pretraining for SLR by providing computationally efficient methods that achieve significant performance improvements across diverse datasets.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
