Ant-inspired Walling Strategies for Scalable Swarm Separation: Reinforcement Learning Approaches Based on Finite State Machines

TL;DR

This paper introduces two decentralized, ant-inspired controllers for robotic swarms to maintain spatial separation, with the DQN-enhanced approach significantly improving adaptability and reducing mixing in simulations.

Contribution

It presents a novel combination of finite-state machines and deep reinforcement learning for scalable swarm separation strategies.

Findings

DQN-enhanced controller reduces mixing by 40-50%.

Both controllers effectively prevent subgroup interference.

Simulation results show faster convergence with DQN approach.

Abstract

In natural systems, emergent structures often arise to balance competing demands. Army ants, for example, form temporary "walls" that prevent interference between foraging trails. Inspired by this behavior, we developed two decentralized controllers for heterogeneous robotic swarms to maintain spatial separation while executing concurrent tasks. The first is a finite-state machine (FSM)-based controller that uses encounter-triggered transitions to create rigid, stable walls. The second integrates FSM states with a Deep Q-Network (DQN), dynamically optimizing separation through emergent "demilitarized zones." In simulation, both controllers reduce mixing between subgroups, with the DQN-enhanced controller improving adaptability and reducing mixing by 40-50% while achieving faster convergence.