Exploiting Ensemble Learning for Cross-View Isolated Sign Language Recognition

TL;DR

This paper presents an ensemble learning approach using a Video Swin Transformer to improve cross-view sign language recognition, addressing viewpoint variability and achieving top competition rankings.

Contribution

It introduces a novel ensemble strategy with a multi-dimensional Video Swin Transformer for cross-view sign language recognition, enhancing robustness and generalization.

Findings

Achieved 3rd place in both RGB and RGB-D tracks at WWW 2025 challenge.

Demonstrated improved cross-view recognition performance.

Validated effectiveness of ensemble learning in sign language recognition.

Abstract

In this paper, we present our solution to the Cross-View Isolated Sign Language Recognition (CV-ISLR) challenge held at WWW 2025. CV-ISLR addresses a critical issue in traditional Isolated Sign Language Recognition (ISLR), where existing datasets predominantly capture sign language videos from a frontal perspective, while real-world camera angles often vary. To accurately recognize sign language from different viewpoints, models must be capable of understanding gestures from multiple angles, making cross-view recognition challenging. To address this, we explore the advantages of ensemble learning, which enhances model robustness and generalization across diverse views. Our approach, built on a multi-dimensional Video Swin Transformer model, leverages this ensemble strategy to achieve competitive performance. Finally, our solution ranked 3rd in both the RGB-based ISLR and RGB-D-based ISLR tracks, demonstrating the effectiveness in handling the challenges of cross-view recognition. The code is available at: https://github.com/Jiafei127/CV_ISLR_WWW2025.