Sensor to Pixels: Decentralized Swarm Gathering via Image-Based Reinforcement Learning

TL;DR

This paper introduces an image-based reinforcement learning approach for decentralized multi-agent swarm control, enabling scalable and efficient aggregation behavior with high convergence rates.

Contribution

It presents a novel image-based reinforcement learning method that encodes observations as visual inputs for decentralized multi-agent control, improving scalability and performance.

Findings

Achieves high convergence rates comparable to neural network benchmarks.

Outperforms traditional analytical solutions in convergence speed.

Effective in limited-range, bearing-only sensing scenarios.

Abstract

This study highlights the potential of image-based reinforcement learning methods for addressing swarm-related tasks. In multi-agent reinforcement learning, effective policy learning depends on how agents sense, interpret, and process inputs. Traditional approaches often rely on handcrafted feature extraction or raw vector-based representations, which limit the scalability and efficiency of learned policies concerning input order and size. In this work we propose an image-based reinforcement learning method for decentralized control of a multi-agent system, where observations are encoded as structured visual inputs that can be processed by Neural Networks, extracting its spatial features and producing novel decentralized motion control rules. We evaluate our approach on a multi-agent convergence task of agents with limited-range and bearing-only sensing that aim to keep the swarm cohesive during the aggregation. The algorithm's performance is evaluated against two benchmarks: an analytical solution proposed by Bellaiche and Bruckstein, which ensures convergence but progresses slowly, and VariAntNet, a neural network-based framework that converges much faster but shows medium success rates in hard constellations. Our method achieves high convergence, with a pace nearly matching that of VariAntNet. In some scenarios, it serves as the only practical alternative.