Hierarchical Procedural Framework for Low-latency Robot-Assisted Hand-Object Interaction

TL;DR

This paper introduces a hierarchical procedural framework for low-latency, adaptive robot-assisted hand-object interaction, enabling seamless coordination in tele-operation scenarios with minimal delay.

Contribution

It presents a novel hierarchical control architecture combining RGB-based 3D hand reconstruction and motion primitives for real-time robot assistance.

Findings

Achieved delay of ≤0.3 seconds in tele-interaction.

Validated effectiveness through experimental validation.

Demonstrated potential in healthcare and manufacturing applications.

Abstract

Advances in robotics have been driving the development of human-robot interaction (HRI) technologies. However, accurately perceiving human actions and achieving adaptive control remains a challenge in facilitating seamless coordination between human and robotic movements. In this paper, we propose a hierarchical procedural framework to enable dynamic robot-assisted hand-object interaction (HOI). An open-loop hierarchy leverages the RGB-based 3D reconstruction of the human hand, based on which motion primitives have been designed to translate hand motions into robotic actions. The low-level coordination hierarchy fine-tunes the robot's action by using the continuously updated 3D hand models. Experimental validation demonstrates the effectiveness of the hierarchical control architecture. The adaptive coordination between human and robot behavior has achieved a delay of $\leq 0.3$ seconds in the tele-interaction scenario. A case study of ring-wearing tasks indicates the potential application of this work in assistive technologies such as healthcare and manufacturing.