VR-Based Control of Multi-Copter Operation

TL;DR

This paper introduces a VR-based teleoperation system for multi-copter control that enhances obstacle awareness and safety in unmapped indoor environments by providing a third-person view and live 3D surroundings reconstruction.

Contribution

The work presents a novel VR teleoperation system with integrated 3D environment reconstruction and compares its performance to traditional FPV control in unmapped spaces.

Findings

TPV maintains task time similar to FPV

Proximal obstacle awareness improves (+0.20m minimum distance)

Contacts are reduced during teleoperation

Abstract

We present a VR-based teleoperation system for multirotor flight that renders a third-person view (TPV) of the vehicle together with a live 3D reconstruction of its surroundings. The system runs on an embedded GPU (Jetson Orin NX) with ROS2-WebXR integration and streams geometry and video to a headset for closed-loop control in previously unmapped spaces. We implement a first-person video (FPV) baseline and perform matched trials with two pilots in unmapped indoor spaces. Quantitative metrics are reported from repeated trials with one pilot (N=8). TPV achieved task time comparable to FPV while improving proximal obstacle awareness (minimum obstacle distance +0.20m) and reducing contacts. These results indicate that TPV can preserve control quality while exposing hazards less visible in FPV, supporting safer teleoperation in unknown environments.