Robot to Human Object Handover using Vision and Joint Torque Sensor Modalities

TL;DR

This paper introduces an autonomous robot-to-human object handover system using vision and joint torque sensors, achieving high accuracy without explicit communication.

Contribution

The work presents a novel sensor fusion approach with deep neural networks for implicit, real-time robot-to-human handover, demonstrating robustness and high accuracy.

Findings

Achieved 98% accuracy in real experiments

Utilized deep neural networks for intention and grasp detection

Enabled fully autonomous, implicit handover without communication

Abstract

We present a robot-to-human object handover algorithm and implement it on a 7-DOF arm equipped with a 3-finger mechanical hand. The system performs a fully autonomous and robust object handover to a human receiver in real-time. Our algorithm relies on two complementary sensor modalities: joint torque sensors on the arm and an eye-in-hand RGB-D camera for sensor feedback. Our approach is entirely implicit, i.e., there is no explicit communication between the robot and the human receiver. Information obtained via the aforementioned sensor modalities is used as inputs to their related deep neural networks. While the torque sensor network detects the human receiver's "intention" such as: pull, hold, or bump, the vision sensor network detects if the receiver's fingers have wrapped around the object. Networks' outputs are then fused, based on which a decision is made to either release the object or not. Despite substantive challenges in sensor feedback synchronization, object, and human hand detection, our system achieves robust robot-to-human handover with 98\% accuracy in our preliminary real experiments using human receivers.