DTPPO: Dual-Transformer Encoder-based Proximal Policy Optimization for Multi-UAV Navigation in Unseen Complex Environments

TL;DR

This paper introduces DTPPO, a dual-transformer based reinforcement learning method that significantly improves multi-UAV navigation in unseen complex environments by modeling inter-agent and temporal dynamics.

Contribution

The paper proposes a novel dual-transformer architecture within PPO that enhances generalization and collaboration among UAVs in complex, unseen environments.

Findings

DTPPO outperforms existing MADRL methods in transferability and obstacle avoidance.

DTPPO demonstrates superior navigation efficiency across diverse obstacle densities.

The architecture enables UAVs to navigate unseen environments without retraining.

Abstract

Existing multi-agent deep reinforcement learning (MADRL) methods for multi-UAV navigation face challenges in generalization, particularly when applied to unseen complex environments. To address these limitations, we propose a Dual-Transformer Encoder-based Proximal Policy Optimization (DTPPO) method. DTPPO enhances multi-UAV collaboration through a Spatial Transformer, which models inter-agent dynamics, and a Temporal Transformer, which captures temporal dependencies to improve generalization across diverse environments. This architecture allows UAVs to navigate new, unseen environments without retraining. Extensive simulations demonstrate that DTPPO outperforms current MADRL methods in terms of transferability, obstacle avoidance, and navigation efficiency across environments with varying obstacle densities. The results confirm DTPPO's effectiveness as a robust solution for multi-UAV navigation in both known and unseen scenarios.