Path Planning through Multi-Agent Reinforcement Learning in Dynamic Environments

TL;DR

This paper introduces a scalable, region-aware multi-agent reinforcement learning framework for dynamic path planning, effectively adapting to local environmental changes and outperforming traditional methods in complex, real-world scenarios.

Contribution

It presents a hierarchical, decentralized RL approach with federated learning for efficient, scalable path planning in dynamic environments, addressing limitations of global planning methods.

Findings

Federated Q-learning outperforms single-agent Q-learning.

Approach closely matches A* Oracle performance.

Decentralized framework improves scalability and adaptation speed.

Abstract

Path planning in dynamic environments is a fundamental challenge in intelligent transportation and robotics, where obstacles and conditions change over time, introducing uncertainty and requiring continuous adaptation. While existing approaches often assume complete environmental unpredictability or rely on global planners, these assumptions limit scalability and practical deployment in real-world settings. In this paper, we propose a scalable, region-aware reinforcement learning (RL) framework for path planning in dynamic environments. Our method builds on the observation that environmental changes, although dynamic, are often localized within bounded regions. To exploit this, we introduce a hierarchical decomposition of the environment and deploy distributed RL agents that adapt to changes locally. We further propose a retraining mechanism based on sub-environment success rates to determine when policy updates are necessary. Two training paradigms are explored: single-agent Q-learning and multi-agent federated Q-learning, where local Q-tables are aggregated periodically to accelerate the learning process. Unlike prior work, we evaluate our methods in more realistic settings, where multiple simultaneous obstacle changes and increasing difficulty levels are present. Results show that the federated variants consistently outperform their single-agent counterparts and closely approach the performance of A* Oracle while maintaining shorter adaptation times and robust scalability. Although initial training remains time-consuming in large environments, our decentralized framework eliminates the need for a global planner and lays the groundwork for future improvements using deep RL and flexible environment decomposition.