DVRP-MHSI: Dynamic Visualization Research Platform for Multimodal Human-Swarm Interaction

TL;DR

This paper introduces DVRP-MHSI, an open, real-time visualization platform supporting multimodal human-swarm interaction, enabling flexible research on swarm algorithms and human control methods through diverse interfaces.

Contribution

The paper presents a novel, versatile platform that integrates multimodal interaction modalities with real-time visualization for human-swarm interaction research.

Findings

Platform effectively visualizes swarm behavior in real-time

Supports multiple interaction modalities including brain-computer and eye-tracking

Validated through demonstrative experiments showing usability and potential

Abstract

In recent years, there has been a significant amount of research on algorithms and control methods for distributed collaborative robots. However, the emergence of collective behavior in a swarm is still difficult to predict and control. Nevertheless, human interaction with the swarm helps render the swarm more predictable and controllable, as human operators can utilize intuition or knowledge that is not always available to the swarm. Therefore, this paper designs the Dynamic Visualization Research Platform for Multimodal Human-Swarm Interaction (DVRP-MHSI), which is an innovative open system that can perform real-time dynamic visualization and is specifically designed to accommodate a multitude of interaction modalities (such as brain-computer, eye-tracking, electromyographic, and touch-based interfaces), thereby expediting progress in human-swarm interaction research. Specifically, the platform consists of custom-made low-cost omnidirectional wheeled mobile robots, multitouch screens and two workstations. In particular, the mutitouch screens can recognize human gestures and the shapes of objects placed on them, and they can also dynamically render diverse scenes. One of the workstations processes communication information within robots and the other one implements human-robot interaction methods. The development of DVRP-MHSI frees researchers from hardware or software details and allows them to focus on versatile swarm algorithms and human-swarm interaction methods without being limited to fixed scenarios, tasks, and interfaces. The effectiveness and potential of the platform for human-swarm interaction studies are validated by several demonstrative experiments.