Strategic Coordination of Drones via Short-term Distributed Optimization and Long-term Reinforcement Learning

TL;DR

This paper introduces a hybrid approach combining long-term reinforcement learning with short-term collective learning to optimize autonomous drone swarms for dynamic environment task allocation, improving efficiency and scalability.

Contribution

It proposes a novel hybrid optimization framework that integrates DRL and collective learning for decentralized drone coordination, addressing limitations of existing methods.

Findings

Outperforms state-of-the-art methods by over 23% in efficiency.

Enhances energy efficiency and accuracy in drone-based traffic monitoring.

Demonstrates scalability and adaptability in urban mobility scenarios.

Abstract

This paper addresses the problem of autonomous task allocation by a swarm of autonomous, interactive drones in large-scale, dynamic spatio-temporal environments. When each drone independently determines navigation, sensing, and recharging options to choose from such that system-wide sensing requirements are met, the collective decision-making becomes an NP-hard decentralized combinatorial optimization problem. Existing solutions face significant limitations: distributed optimization methods such as collective learning often lack long-term adaptability, while centralized deep reinforcement learning (DRL) suffers from high computational complexity, scalability and privacy concerns. To overcome these challenges, we propose a novel hybrid optimization approach that combines long-term DRL with short-term collective learning. In this approach, each drone uses DRL methods to proactively determine high-level strategies, such as flight direction and recharging behavior, while leveraging collective learning to coordinate short-term sensing and navigation tasks with other drones in a decentralized manner. Extensive experiments using datasets derived from realistic urban mobility demonstrate that the proposed solution outperforms standalone state-of-the-art collective learning and DRL approaches by $27.83\%$ and $23.17\%$ respectively. Our findings highlight the complementary strengths of short-term and long-term decision-making, enabling energy-efficient, accurate, and sustainable traffic monitoring through swarms of drones.