Compact Real-time avoidance on a Humanoid Robot for Human-robot Interaction

TL;DR

This paper presents a real-time, self-contained framework for humanoid robots to dynamically maintain a safety zone around themselves using deep learning-based human keypoint estimation, disparity, and obstacle avoidance, enhancing human-robot interaction safety.

Contribution

It introduces a novel, real-time safety zone maintenance system for humanoid robots that integrates 2D/3D human keypoints, peripersonal space representation, and obstacle avoidance without external sensors.

Findings

Effective safety margins maintained in pilot tests.

Flexible safety zones modulated around different body parts.

System operates in real time using onboard cameras.

Abstract

With robots leaving factories and entering less controlled domains, possibly sharing the space with humans, safety is paramount and multimodal awareness of the body surface and the surrounding environment is fundamental. Taking inspiration from peripersonal space representations in humans, we present a framework on a humanoid robot that dynamically maintains such a protective safety zone, composed of the following main components: (i) a human 2D keypoints estimation pipeline employing a deep learning based algorithm, extended here into 3D using disparity; (ii) a distributed peripersonal space representation around the robot's body parts; (iii) a reaching controller that incorporates all obstacles entering the robot's safety zone on the fly into the task. Pilot experiments demonstrate that an effective safety margin between the robot's and the human's body parts is kept. The proposed solution is flexible and versatile since the safety zone around individual robot and human body parts can be selectively modulated---here we demonstrate stronger avoidance of the human head compared to rest of the body. Our system works in real time and is self-contained, with no external sensory equipment and use of onboard cameras only.