Teleoperated aerial manipulator and its avatar. Part 1: Communication, system's interconnection, control, and virtual world

TL;DR

This paper presents a teleoperated aerial manipulator system integrating virtual reality and internet-based control, demonstrated through software-in-the-loop simulations, aiming to enable complex tasks via human-in-the-loop control.

Contribution

It introduces a novel integrated system combining aerial manipulator modeling, virtual reality immersion, and remote control over the internet, with experiments in simulation.

Findings

Successful implementation of teleoperation system in simulation

Effective integration of virtual reality with aerial manipulator control

Open-source code and demonstration videos provided

Abstract

The tasks that an aerial manipulator can perform are incredibly diverse. However, nowadays the technology is not completely developed to achieve complex tasks autonomously. That's why we propose a human-in-the-loop system that can control a semi-autonomous aerial manipulator to accomplish these kinds of tasks. Furthermore, motivated by the growing trend of virtual reality systems, together with teleoperation, we develop a system composed of: an aerial manipulator model programmed in PX4 and modeled in Gazebo, a virtual reality immersion with an interactive controller, and the interconnection between the aforementioned systems via the Internet. This research is the first part of a broader project. In this part, we present experiments in the software in the loop simulation. The code of this work is liberated on our GitHub page. Also, a video shows the conducted experiments.