Virtual Reality Based Robot Teleoperation via Human-Scene Interaction

TL;DR

This paper introduces a VR-based robot teleoperation system that enhances user interaction, reduces workload, and improves efficiency in remote manipulation tasks through intuitive interfaces and scene simulation.

Contribution

It presents a novel VR teleoperation system with a user-friendly interface and scene preview simulation, improving performance and user experience over traditional methods.

Findings

Users perform tasks faster with VR system

Reduced mental workload in teleoperation

System outperforms direct control methods

Abstract

Robot teleoperation gains great success in various situations, including chemical pollution rescue, disaster relief, and long-distance manipulation. In this article, we propose a virtual reality (VR) based robot teleoperation system to achieve more efficient and natural interaction with humans in different scenes. A user-friendly VR interface is designed to help users interact with a desktop scene using their hands efficiently and intuitively. To improve user experience and reduce workload, we simulate the process in the physics engine to help build a preview of the scene after manipulation in the virtual scene before execution. We conduct experiments with different users and compare our system with a direct control method across several teleoperation tasks. The user study demonstrates that the proposed system enables users to perform operations more instinctively with a lighter mental workload. Users can perform pick-and-place and object-stacking tasks in a considerably short time, even for beginners. Our code is available at https://github.com/lingxiaomeng/VR_Teleoperation_Gen3.