Learn from Human Teams: a Probabilistic Solution to Real-Time Collaborative Robot Handling with Dynamic Gesture Commands

TL;DR

This paper introduces RTCoHand, a probabilistic framework enabling real-time, gesture-based control of collaborative robots, improving flexibility, safety, and user experience in handling tasks under human commands.

Contribution

The paper presents a novel probabilistic model for gesture-based robot control that handles user variability and noise, advancing real-time collaborative robot handling.

Findings

RTCoHand enables complex maneuvers with lower cognitive load.

The approach improves safety and reduces trial-and-error in robot handling.

It demonstrates robustness across different users and conditions.

Abstract

We study real-time collaborative robot (cobot) handling, where the cobot maneuvers a workpiece under human commands. This is useful when it is risky for humans to directly handle the workpiece. However, it is hard to make the cobot both easy to command and flexible in possible operations. In this work, we propose a Real-Time Collaborative Robot Handling (RTCoHand) framework that allows the control of cobot via user-customized dynamic gestures. This is hard due to variations among users, human motion uncertainties, and noisy human input. We model the task as a probabilistic generative process, referred to as Conditional Collaborative Handling Process (CCHP), and learn from human-human collaboration. We thoroughly evaluate the adaptability and robustness of CCHP and apply our approach to a real-time cobot handling task with Kinova Gen3 robot arm. We achieve seamless human-robot collaboration with both experienced and new users. Compared to classical controllers, RTCoHand allows significantly more complex maneuvers and lower user cognitive burden. It also eliminates the need for trial-and-error, rendering it advantageous in safety-critical tasks.