NimbRo Avatar: Interactive Immersive Telepresence with Force-Feedback Telemanipulation

TL;DR

This paper presents NimbRo Avatar, an immersive telepresence system with force-feedback telemanipulation, 3D visualization, and autonomous locomotion, enabling realistic remote interaction and manipulation in indoor environments.

Contribution

The paper introduces a novel immersive telepresence robot with force-feedback, dexterous manipulation, and integrated visualization, advancing remote human-robot interaction capabilities.

Findings

Successful user study with untrained operators

Effective long-duration complex mission execution

Positive feedback on system usability and immersion

Abstract

Robotic avatars promise immersive teleoperation with human-like manipulation and communication capabilities. We present such an avatar system, based on the key components of immersive 3D visualization and transparent force-feedback telemanipulation. Our avatar robot features an anthropomorphic bimanual arm configuration with dexterous hands. The remote human operator drives the arms and fingers through an exoskeleton-based operator station, which provides force feedback both at the wrist and for each finger. The robot torso is mounted on a holonomic base, providing locomotion capability in typical indoor scenarios, controlled using a 3D rudder device. Finally, the robot features a 6D movable head with stereo cameras, which stream images to a VR HMD worn by the operator. Movement latency is hidden using spherical rendering. The head also carries a telepresence screen displaying a synthesized image of the operator with facial animation, which enables direct interaction with remote persons. We evaluate our system successfully both in a user study with untrained operators as well as a longer and more complex integrated mission. We discuss lessons learned from the trials and possible improvements.