Towards safe human-to-robot handovers of unknown containers

TL;DR

This paper introduces a simulation-based framework for safe human-to-robot handovers of unknown containers, estimating object and hand properties from videos to ensure safety without physical testing.

Contribution

It presents a novel real-to-simulation approach that estimates physical properties and hand poses from videos, enabling safe handovers without expensive equipment or risk to humans or robots.

Findings

Framework accurately estimates object properties from videos.

Simulation-based handover validation ensures safety.

Noisy perceptual estimates still maintain safeness.

Abstract

Safe human-to-robot handovers of unknown objects require accurate estimation of hand poses and object properties, such as shape, trajectory, and weight. Accurately estimating these properties requires the use of scanned 3D object models or expensive equipment, such as motion capture systems and markers, or both. However, testing handover algorithms with robots may be dangerous for the human and, when the object is an open container with liquids, for the robot. In this paper, we propose a real-to-simulation framework to develop safe human-to-robot handovers with estimations of the physical properties of unknown cups or drinking glasses and estimations of the human hands from videos of a human manipulating the container. We complete the handover in simulation, and we estimate a region that is not occluded by the hand of the human holding the container. We also quantify the safeness of the human and object in simulation. We validate the framework using public recordings of containers manipulated before a handover and show the safeness of the handover when using noisy estimates from a range of perceptual algorithms.