DroneTrap: Drone Catching in Midair by Soft Robotic Hand with Color-Based Force Detection and Hand Gesture Recognition

TL;DR

DroneTrap introduces a soft robotic hand with color-based force detection and gesture recognition, enabling safe, precise midair drone catching controlled by human gestures for various practical applications.

Contribution

The paper presents a novel soft robotic hand with integrated color-based force estimation and gesture control, enhancing drone docking safety and efficiency.

Findings

Color-based force estimation achieves 95.3% accuracy.

Real-time gesture control enables intuitive robot operation.

Potential applications include drone delivery and inspection.

Abstract

The paper proposes a novel concept of docking drones to make this process as safe and fast as possible. The idea behind the project is that a robot with a soft gripper grasps the drone in midair. The human operator navigates the robotic arm with the ML-based gesture recognition interface. The 3-finger robot hand with soft fingers is equipped with touch sensors, making it possible to achieve safe drone catching and avoid inadvertent damage to the drone's propellers and motors. Additionally, the soft hand is featured with a unique color-based force estimation technology based on a computer vision (CV) system. Moreover, the visual color-changing system makes it easier for the human operator to interpret the applied forces. Without any additional programming, the operator has full real-time control of the robot's motion and task execution by wearing a mocap glove with gesture recognition, which was developed and applied for the high-level control of DroneTrap. The experimental results revealed that the developed color-based force estimation can be applied for rigid object capturing with high precision (95.3\%). The proposed technology can potentially revolutionize the landing and deployment of drones for parcel delivery on uneven ground, structure maintenance and inspection, risque operations, and etc.