Towards Decentralized Human-Swarm Interaction by Means of Sequential Hand Gesture Recognition
Zahi Kakish, Sritanay Vedartham, Spring Berman

TL;DR
This paper introduces a decentralized human-swarm interaction method using hand gesture recognition with CNNs, enabling robots to modify their behavior based on gestures in simulation and real-world tests.
Contribution
It presents a novel approach combining CNN-based gesture recognition with decentralized robot control for human-swarm interaction.
Findings
Successful recognition of hand gestures in simulation and real-world environments.
Effective modification of swarm behavior based on gesture sequences.
Validation in Gazebo simulations and physical robot tests.
Abstract
In this work, we present preliminary work on a novel method for Human-Swarm Interaction (HSI) that can be used to change the macroscopic behavior of a swarm of robots with decentralized sensing and control. By integrating a small yet capable hand gesture recognition convolutional neural network (CNN) with the next-generation Robot Operating System \emph{ros2}, which enables decentralized implementation of robot software for multi-robot applications, we demonstrate the feasibility of programming a swarm of robots to recognize and respond to a sequence of hand gestures that capable of correspond to different types of swarm behaviors. We test our approach using a sequence of gestures that modifies the target inter-robot distance in a group of three Turtlebot3 Burger robots in order to prevent robot collisions with obstacles. The approach is validated in three different Gazebo simulation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Robotics and Automated Systems · Robot Manipulation and Learning
