HyperPalm: DNN-based hand gesture recognition interface for intelligent communication with quadruped robot in 3D space

TL;DR

HyperPalm is a DNN-based hand gesture recognition system enabling intuitive 3D control of quadruped robots, achieving high accuracy and low deviation, thus improving human-robot interaction without additional devices.

Contribution

The paper introduces HyperPalm, a novel gesture interface that allows full 6DoF control of quadruped robots using only 5 static gestures and hand motion, with high accuracy.

Findings

Gesture classification accuracy of 96.5%

Position prediction RMSD of 11.7 mm

User experience rated as intuitive and low effort

Abstract

Nowadays, autonomous mobile robots support people in many areas where human presence either redundant or too dangerous. They have successfully proven themselves in expeditions, gas industry, mines, warehouses, etc. However, even legged robots may stuck in rough terrain conditions requiring human cognitive abilities to navigate the system. While gamepads and keyboards are convenient for wheeled robot control, the quadruped robot in 3D space can move along all linear coordinates and Euler angles, requiring at least 12 buttons for independent control of their DoF. Therefore, more convenient interfaces of control are required. In this paper we present HyperPalm: a novel gesture interface for intuitive human-robot interaction with quadruped robots. Without additional devices, the operator has full position and orientation control of the quadruped robot in 3D space through hand gesture recognition with only 5 gestures and 6 DoF hand motion. The experimental results revealed to classify 5 static gestures with high accuracy (96.5%), accurately predict the position of the 6D position of the hand in three-dimensional space. The absolute linear deviation Root mean square deviation (RMSD) of the proposed approach is 11.7 mm, which is almost 50% lower than for the second tested approach, the absolute angular deviation RMSD of the proposed approach is 2.6 degrees, which is almost 27% lower than for the second tested approach. Moreover, the user study was conducted to explore user's subjective experience from human-robot interaction through the proposed gesture interface. The participants evaluated their interaction with HyperPalm as intuitive (2.0), not causing frustration (2.63), and requiring low physical demand (2.0).