GRLib: An Open-Source Hand Gesture Detection and Recognition Python Library

TL;DR

GRLib is an open-source Python library that detects and recognizes static and dynamic hand gestures using RGB camera input, employing data augmentation and MediaPipe Hands, and outperforms existing solutions on real-world datasets.

Contribution

The paper introduces GRLib, a novel open-source library for hand gesture recognition that supports training, dynamic gestures, and demonstrates superior performance over existing solutions.

Findings

Outperforms MediaPipe Solutions on three datasets

Supports training for improved robustness

Handles static and dynamic gestures effectively

Abstract

Hand gesture recognition systems provide a natural way for humans to interact with computer systems. Although various algorithms have been designed for this task, a host of external conditions, such as poor lighting or distance from the camera, make it difficult to create an algorithm that performs well across a range of environments. In this work, we present GRLib: an open-source Python library able to detect and classify static and dynamic hand gestures. Moreover, the library can be trained on existing data for improved classification robustness. The proposed solution utilizes a feed from an RGB camera. The retrieved frames are then subjected to data augmentation and passed on to MediaPipe Hands to perform hand landmark detection. The landmarks are then classified into their respective gesture class. The library supports dynamic hand gestures through trajectories and keyframe extraction. It was found that the library outperforms another publicly available HGR system - MediaPipe Solutions, on three diverse, real-world datasets. The library is available at https://github.com/mikhail-vlasenko/grlib and can be installed with pip.