Observer Design for Augmented Reality-based Teleoperation of Soft Robots

TL;DR

This paper introduces an augmented reality interface for teleoperating soft robots, demonstrating that AR can effectively facilitate interaction and control despite modeling challenges, with errors around 5% of robot length.

Contribution

It presents a novel AR-based teleoperation system for soft robots, integrating physics-based position estimation and validation on a modular pneumatic manipulator.

Findings

AR interface enables effective soft robot control

Position estimation error is approximately 5% of robot length

System integrates AR with control loop for soft robots

Abstract

Although virtual and augmented reality are gaining traction as teleoperation tools for various types of robots, including manipulators and mobile robots, they are not being used for soft robots. The inherent difficulties of modelling soft robots mean that combining accurate and computationally efficient representations is very challenging. This paper presents an augmented reality interface for teleoperating these devices. The developed system consists of Microsoft HoloLens 2 glasses and a central computer responsible for calculations. Validation is performed on PETER, a highly modular pneumatic manipulator. Using data collected from sensors, the computer estimates the robot's position based on the physics of the virtual reality programme. Errors obtained are on the order of 5% of the robot's length, demonstrating that augmented reality facilitates operator interaction with soft manipulators and can be integrated into the control loop.