Human Grasp Classification for Reactive Human-to-Robot Handovers

TL;DR

This paper introduces a novel method for human-to-robot object handovers that classifies human grasps and plans trajectories to improve fluency and usability in collaborative tasks.

Contribution

It presents a new grasp classification dataset, a deep learning model for grasp recognition, and a planning approach for more effective human-to-robot handovers.

Findings

System achieves more fluent handovers than baselines.

User study shows high effectiveness and usability.

System adapts to interrupted handovers with replanning.

Abstract

Transfer of objects between humans and robots is a critical capability for collaborative robots. Although there has been a recent surge of interest in human-robot handovers, most prior research focus on robot-to-human handovers. Further, work on the equally critical human-to-robot handovers often assumes humans can place the object in the robot's gripper. In this paper, we propose an approach for human-to-robot handovers in which the robot meets the human halfway, by classifying the human's grasp of the object and quickly planning a trajectory accordingly to take the object from the human's hand according to their intent. To do this, we collect a human grasp dataset which covers typical ways of holding objects with various hand shapes and poses, and learn a deep model on this dataset to classify the hand grasps into one of these categories. We present a planning and execution approach that takes the object from the human hand according to the detected grasp and hand position, and replans as necessary when the handover is interrupted. Through a systematic evaluation, we demonstrate that our system results in more fluent handovers versus two baselines. We also present findings from a user study (N = 9) demonstrating the effectiveness and usability of our approach with naive users in different scenarios. More results and videos can be found at http://wyang.me/handovers.