Cooperative UAVs for Remote Data Collection under Limited Communications: An Asynchronous Multiagent Learning Framework

TL;DR

This paper proposes an asynchronous multi-agent learning framework for UAVs to optimize trajectories and bandwidth allocation, improving energy efficiency and robustness in remote data collection without requiring synchronized actions.

Contribution

It introduces a novel decentralized semi-Markov decision process formulation and an asynchronous learning algorithm tailored for UAV cooperation in asynchronous environments.

Findings

Outperforms existing methods in energy efficiency and mission time

Demonstrates robustness under varying environmental conditions

Provides an effective solution for asynchronous multi-UAV coordination

Abstract

This paper addresses the joint optimization of trajectories and bandwidth allocation for multiple Unmanned Aerial Vehicles (UAVs) to enhance energy efficiency in the cooperative data collection problem. We focus on an important yet underestimated aspect of the system, where action synchronization across all UAVs is impossible. Since most existing learning-based solutions are not designed to learn in this asynchronous environment, we formulate the trajectory planning problem as a Decentralized Partially Observable Semi-Markov Decision Process and introduce an asynchronous multi-agent learning algorithm to learn UAVs' cooperative policies. Once the UAVs' trajectory policies are learned, the bandwidth allocation can be optimally solved based on local observations at each collection point. Comprehensive empirical results demonstrate the superiority of the proposed method over other learning-based and heuristic baselines in terms of both energy efficiency and mission completion time. Additionally, the learned policies exhibit robustness under varying environmental conditions.