Shared-Control Robotic Manipulation in Virtual Reality

TL;DR

This paper introduces a VR-based teleoperation system for robotic manipulators that combines human-in-the-loop control with visual feedback, enhancing intuitive interaction and task execution in robotic manipulation.

Contribution

It presents a novel VR interface with iterative human-in-the-loop control and real-time environmental feedback for improved robotic teleoperation.

Findings

Successfully demonstrated in pick and place tasks

Provides real-time point cloud and video feedback

Potential for increased fluency and autonomy

Abstract

In this paper, we present the implementation details of a Virtual Reality (VR)-based teleoperation interface for moving a robotic manipulator. We propose an iterative human-in-the-loop design where the user sets the next task-space waypoint for the robot's end effector and executes the action on the physical robot before setting the next waypoints. Information from the robot's surroundings is provided to the user in two forms: as a point cloud in 3D space and a video stream projected on a virtual wall. The feasibility of the selected end effector pose is communicated to the user by the color of the virtual end effector. The interface is demonstrated to successfully work for a pick and place scenario, however, our trials showed that the fluency of the interaction and the autonomy level of the system can be increased.