Intuitive Human-Robot Interface: A 3-Dimensional Action Recognition and UAV Collaboration Framework

TL;DR

This paper presents an intuitive human-robot interface that uses 3D action recognition via stereo cameras and a k-nearest neighbor classifier to enable natural UAV control and tracking based on human movements.

Contribution

It introduces a novel 3D human action classification method combined with UAV coordination, enhancing intuitive control and continuous human tracking.

Findings

Effective 3D action recognition from stereo camera data

Successful UAV coordination based on human gestures

Robust human tracking during UAV operation

Abstract

Harnessing human movements to command an Unmanned Aerial Vehicle (UAV) holds the potential to revolutionize their deployment, rendering it more intuitive and user-centric. In this research, we introduce a novel methodology adept at classifying three-dimensional human actions, leveraging them to coordinate on-field with a UAV. Utilizing a stereo camera, we derive both RGB and depth data, subsequently extracting three-dimensional human poses from the continuous video feed. This data is then processed through our proposed k-nearest neighbour classifier, the results of which dictate the behaviour of the UAV. It also includes mechanisms ensuring the robot perpetually maintains the human within its visual purview, adeptly tracking user movements. We subjected our approach to rigorous testing involving multiple tests with real robots. The ensuing results, coupled with comprehensive analysis, underscore the efficacy and inherent advantages of our proposed methodology.