Personalized Human-Swarm Interaction through Hand Motion

TL;DR

This paper presents a novel personalized hand motion-based interface for drone swarm teleoperation, leveraging machine learning to adapt to individual user preferences and improve control performance.

Contribution

It introduces a machine learning approach to create personalized human-swarm interfaces using hand motion tracking with LEAP Motion, enhancing portability and user experience.

Findings

Effective personalization of control interfaces for drone swarms.

LEAP Motion enables portable and intuitive hand tracking.

Promising results in simulated environments with user preference adaptation.

Abstract

The control of collective robotic systems, such as drone swarms, is often delegated to autonomous navigation algorithms due to their high dimensionality. However, like other robotic entities, drone swarms can still benefit from being teleoperated by human operators, whose perception and decision-making capabilities are still out of the reach of autonomous systems. Drone swarm teleoperation is only at its dawn, and a standard human-swarm interface (HRI) is missing to date. In this study, we analyzed the spontaneous interaction strategies of naive users with a swarm of drones. We implemented a machine-learning algorithm to define a personalized Body-Machine Interface (BoMI) based only on a short calibration procedure. During this procedure, the human operator is asked to move spontaneously as if they were in control of a simulated drone swarm. We assessed that hands are the most commonly adopted body segment, and thus we chose a LEAP Motion controller to track them to let the users control the aerial drone swarm. This choice makes our interface portable since it does not rely on a centralized system for tracking the human body. We validated our algorithm to define personalized HRIs for a set of participants in a realistic simulated environment, showing promising results in performance and user experience. Our method leaves unprecedented freedom to the user to choose between position and velocity control only based on their body motion preferences.