SwarmPaint: Human-Swarm Interaction for Trajectory Generation and Formation Control by DNN-based Gesture Interface
Valerii Serpiva, Ekaterina Karmanova, Aleksey Fedoseev, Stepan, Perminov, and Dzmitry Tsetserukou

TL;DR
SwarmPaint introduces a DNN-based gesture interface enabling intuitive human control of multi-agent swarms for trajectory and formation tasks, validated through user studies showing promising accuracy and user experience.
Contribution
The paper presents a novel gesture-based human-swarm interaction system that allows control without additional devices, combining trajectory and formation control modes.
Findings
Achieved mean trajectory error of 5.6 cm with gesture drawing.
Trajectory drawing interface rated more intuitive and easier to use.
Participants could target within 7.3 cm accuracy on 1 m patterns.
Abstract
Teleoperation tasks with multi-agent systems have a high potential in supporting human-swarm collaborative teams in exploration and rescue operations. However, it requires an intuitive and adaptive control approach to ensure swarm stability in a cluttered and dynamically shifting environment. We propose a novel human-swarm interaction system, allowing the user to control swarm position and formation by either direct hand motion or by trajectory drawing with a hand gesture interface based on the DNN gesture recognition. The key technology of the SwarmPaint is the user's ability to perform various tasks with the swarm without additional devices by switching between interaction modes. Two types of interaction were proposed and developed to adjust a swarm behavior: free-form trajectory generation control and shaped formation control. Two preliminary user studies were conducted to explore…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
