Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing

TL;DR

This paper develops a robust multi-agent reinforcement learning approach to ensure small UAS separation safety under GPS degradation and spoofing, using a novel adversarial perturbation model and demonstrating high safety performance in simulations.

Contribution

It introduces a closed-form adversarial perturbation expression for robust MARL, enabling efficient evaluation and improved safety under GPS spoofing conditions.

Findings

Near-zero collision rates up to 35% GPS corruption

Closed-form adversarial perturbation approximates worst-case with second-order accuracy

Robust policy outperforms baseline in high-density UAS simulations

Abstract

We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) broadcasts its GPS-derived position; when such position broadcasts are corrupted, the entire observed air traffic state becomes unreliable. We cast this state observation corruption as a zero-sum game between the agents and an adversary: with probability R, the adversary perturbs the observed state to maximally degrade each agent's safety performance. We derive a closed-form expression for this adversarial perturbation, bypassing adversarial training entirely and enabling linear-time evaluation in the state dimension. We show that this expression approximates the true worst-case adversarial perturbation with second-order accuracy. We further bound the safety performance gap between clean and corrupted observations, showing that it degrades at most linearly with the corruption probability under Kullback-Leibler regularization. Finally, we integrate the closed-form adversarial policy into a MARL policy gradient algorithm to obtain a robust counter-policy for the agents. In a high-density sUAS simulation, we observe near-zero collision rates under corruption levels up to 35%, outperforming a baseline policy trained without adversarial perturbations.