CoDe: A Cooperative and Decentralized Collision Avoidance Algorithm for Small-Scale UAV Swarms Considering Energy Efficiency

TL;DR

CoDe is a novel cooperative decentralized collision avoidance algorithm for small UAV swarms that enhances energy efficiency and addresses key challenges in multi-agent reinforcement learning.

Contribution

It introduces a new credit assignment scheme within MARL tailored for UAVs, improving cooperation and computational efficiency.

Findings

Outperforms existing MARL and heuristic algorithms in experiments.

Enhances energy efficiency and cooperation among UAVs.

Addresses continuous action space challenges in UAV collision avoidance.

Abstract

This paper introduces a cooperative and decentralized collision avoidance algorithm (CoDe) for small-scale UAV swarms consisting of up to three UAVs. CoDe improves energy efficiency of UAVs by achieving effective cooperation among UAVs. Moreover, CoDe is specifically tailored for UAV's operations by addressing the challenges faced by existing schemes, such as ineffectiveness in selecting actions from continuous action spaces and high computational complexity. CoDe is based on Multi-Agent Reinforcement Learning (MARL), and finds cooperative policies by incorporating a novel credit assignment scheme. The novel credit assignment scheme estimates the contribution of an individual by subtracting a baseline from the joint action value for the swarm. The credit assignment scheme in CoDe outperforms other benchmarks as the baseline takes into account not only the importance of a UAV's action but also the interrelation between UAVs. Furthermore, extensive experiments are conducted against existing MARL-based and conventional heuristic-based algorithms to demonstrate the advantages of the proposed algorithm.