Hand Gesture Recognition for Collaborative Robots Using Lightweight Deep Learning in Real-Time Robotic Systems

TL;DR

This paper introduces a highly lightweight deep learning system for real-time hand gesture recognition that enables natural human-robot interaction with minimal model size and high accuracy, suitable for deployment on edge devices.

Contribution

The paper presents a novel ultra-lightweight deep learning model for hand gesture recognition that is optimized for real-time control of collaborative robots on resource-constrained devices.

Findings

Achieved 93.5% accuracy with only 1,103 parameters.

Reduced model size to 7 KB using quantization and pruning.

Successfully integrated and tested on a UR5 robot with ROS2.

Abstract

Direct and natural interaction is essential for intuitive human-robot collaboration, eliminating the need for additional devices such as joysticks, tablets, or wearable sensors. In this paper, we present a lightweight deep learning-based hand gesture recognition system that enables humans to control collaborative robots naturally and efficiently. This model recognizes eight distinct hand gestures with only 1,103 parameters and a compact size of 22 KB, achieving an accuracy of 93.5%. To further optimize the model for real-world deployment on edge devices, we applied quantization and pruning using TensorFlow Lite, reducing the final model size to just 7 KB. The system was successfully implemented and tested on a Universal Robot UR5 collaborative robot within a real-time robotic framework based on ROS2. The results demonstrate that even extremely lightweight models can deliver accurate and responsive hand gesture-based control for collaborative robots, opening new possibilities for natural human-robot interaction in constrained environments.