Shared Telemanipulation with VR controllers in an anti slosh scenario

TL;DR

This paper introduces a low-cost VR controller-based telemanipulation framework using nonlinear model predictive control, enhancing transparency and feedback for tasks like liquid handling with an UR5e robot.

Contribution

It presents a novel telemanipulation approach combining low-cost VR controllers with nonlinear model predictive control for improved task performance.

Findings

Successful liquid handling without spilling using VR controllers

Framework validated on UR5e robot arm

Extensions like pouring assistance and collision avoidance feasible

Abstract

Telemanipulation has become a promising technology that combines human intelligence with robotic capabilities to perform tasks remotely. However, it faces several challenges such as insufficient transparency, low immersion, and limited feedback to the human operator. Moreover, the high cost of haptic interfaces is a major limitation for the application of telemanipulation in various fields, including elder care, where our research is focused. To address these challenges, this paper proposes the usage of nonlinear model predictive control for telemanipulation using low-cost virtual reality controllers, including multiple control goals in the objective function. The framework utilizes models for human input prediction and taskrelated models of the robot and the environment. The proposed framework is validated on an UR5e robot arm in the scenario of handling liquid without spilling. Further extensions of the framework such as pouring assistance and collision avoidance can easily be included.