HaGRID - HAnd Gesture Recognition Image Dataset

TL;DR

HaGRID is a large, diverse hand gesture image dataset designed to improve gesture recognition systems by including static and dynamic gestures, with extensive annotations and real-world variability.

Contribution

The paper introduces HaGRID, a comprehensive dataset with over half a million images capturing diverse static and dynamic hand gestures for improved recognition models.

Findings

HaGRID enables effective training of gesture recognition models.

Diversity in subjects and conditions enhances model robustness.

Pretraining on HaGRID improves recognition accuracy.

Abstract

This paper introduces an enormous dataset, HaGRID (HAnd Gesture Recognition Image Dataset), to build a hand gesture recognition (HGR) system concentrating on interaction with devices to manage them. That is why all 18 chosen gestures are endowed with the semiotic function and can be interpreted as a specific action. Although the gestures are static, they were picked up, especially for the ability to design several dynamic gestures. It allows the trained model to recognize not only static gestures such as "like" and "stop" but also "swipes" and "drag and drop" dynamic gestures. The HaGRID contains 554,800 images and bounding box annotations with gesture labels to solve hand detection and gesture classification tasks. The low variability in context and subjects of other datasets was the reason for creating the dataset without such limitations. Utilizing crowdsourcing platforms allowed us to collect samples recorded by 37,583 subjects in at least as many scenes with subject-to-camera distances from 0.5 to 4 meters in various natural light conditions. The influence of the diversity characteristics was assessed in ablation study experiments. Also, we demonstrate the HaGRID ability to be used for pretraining models in HGR tasks. The HaGRID and pretrained models are publicly available.