Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments

TL;DR

This paper introduces a VR-based shared control system for drone teams in unknown environments, enhancing navigation safety and operator influence through real-time, user-guided planning and control.

Contribution

It presents a novel immersive shared control framework combining motion primitives and bilateral VR interface for human-guided multi-robot exploration.

Findings

Shared control improves obstacle avoidance and inter-agent spacing.

Operator effort is reduced with the VR interface.

System supports mixed-reality operations with physical and simulated drones.

Abstract

While autonomous multi-robots can achieve safe and coordinated navigation, they often struggle to adapt to unforeseen conditions and to capture operator-driven objectives in unstructured environments. We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments, enabling real-time, user-guided exploration. At the core of our approach is a novel, user-guided motion-primitive-based planner that computes continuous, collision-free trajectories while continuously integrating operator input. This planner is coupled with an admittance controller, allowing the operator to flexibly influence team behavior and guide drones toward regions of interest that autonomous planners may overlook. The system supports mixed-reality operations with both physical and simulated drones, and implements a bilateral VR-based interface, allowing the operator to guide the robot team via migration points while receiving immediate visual feedback of the team state. Experimental results show that shared control improves obstacle avoidance, maintains inter-agent spacing, and reduces operator effort, demonstrating the feasibility and advantages of immersive, human-in-the-loop multi-robot navigation.