Long Short-Term Memory for Spatial Encoding in Multi-Agent Path Planning

TL;DR

This paper introduces a reinforcement learning approach with LSTM-based spatial encoding for multi-agent path planning, enabling scalable guidance for an indefinite number of agents in real-world drone navigation.

Contribution

It proposes a novel LSTM-based policy architecture that scales to unlimited agents and physical dimensions, validated through real drone flight tests.

Findings

Successfully guided up to four drones collision-free in real-world tests

Scalable guidance system applicable to an indefinite number of agents

Real-time implementation on low-cost onboard hardware

Abstract

Reinforcement learning-based path planning for multi-agent systems of varying size constitutes a research topic with increasing significance as progress in domains such as urban air mobility and autonomous aerial vehicles continues. Reinforcement learning with continuous state and action spaces is used to train a policy network that accommodates desirable path planning behaviors and can be used for time-critical applications. A Long Short-Term Memory module is proposed to encode an unspecified number of states for a varying, indefinite number of agents. The described training strategies and policy architecture lead to a guidance that scales to an infinite number of agents and unlimited physical dimensions, although training takes place at a smaller scale. The guidance is implemented on a low-cost, off-the-shelf onboard computer. The feasibility of the proposed approach is validated by presenting flight test results of up to four drones, autonomously navigating collision-free in a real-world environment.