Signformer is all you need: Towards Edge AI for Sign Language

TL;DR

Signformer introduces a from-scratch, efficient transformer model for sign language translation that achieves high performance without relying on pretrained models, significantly reducing parameters and enabling edge deployment.

Contribution

This paper presents Signformer, a novel from-scratch transformer architecture for sign language translation that outperforms state-of-the-art models in efficiency and scalability without external pretrained resources.

Findings

Achieves 2nd place on leaderboard with significantly fewer parameters.

Reduces model size by 467-1807x compared to top models.

Outperforms other methods with only 0.57 million parameters.

Abstract

Sign language translation, especially in gloss-free paradigm, is confronting a dilemma of impracticality and unsustainability due to growing resource-intensive methodologies. Contemporary state-of-the-arts (SOTAs) have significantly hinged on pretrained sophiscated backbones such as Large Language Models (LLMs), embedding sources, or extensive datasets, inducing considerable parametric and computational inefficiency for sustainable use in real-world scenario. Despite their success, following this research direction undermines the overarching mission of this domain to create substantial value to bridge hard-hearing and common populations. Committing to the prevailing trend of LLM and Natural Language Processing (NLP) studies, we pursue a profound essential change in architecture to achieve ground-up improvements without external aid from pretrained models, prior knowledge transfer, or any NLP strategies considered not-from-scratch. Introducing Signformer, a from-scratch Feather-Giant transforming the area towards Edge AI that redefines extremities of performance and efficiency with LLM-competence and edgy-deployable compactness. In this paper, we present nature analysis of sign languages to inform our algorithmic design and deliver a scalable transformer pipeline with convolution and attention novelty. We achieve new 2nd place on leaderboard with a parametric reduction of 467-1807x against the finests as of 2024 and outcompete almost every other methods in a lighter configuration of 0.57 million parameters.