ARMOR: Robust Reinforcement Learning-based Control for UAVs under Physical Attacks

TL;DR

ARMOR is a novel attack-resilient reinforcement learning controller for UAVs that learns robust state representations to maintain safe operation under physical sensor attacks, outperforming existing methods.

Contribution

Introduces ARMOR, a model-free RL framework that uses a two-stage training process to achieve attack resilience without relying on privileged attack information.

Findings

ARMOR outperforms conventional methods in attack scenarios.

It generalizes better to unseen attacks.

Reduces training cost by avoiding iterative adversarial training.

Abstract

Unmanned Aerial Vehicles (UAVs) depend on onboard sensors for perception, navigation, and control. However, these sensors are susceptible to physical attacks, such as GPS spoofing, that can corrupt state estimates and lead to unsafe behavior. While reinforcement learning (RL) offers adaptive control capabilities, existing safe RL methods are ineffective against such attacks. We present ARMOR (Adaptive Robust Manipulation-Optimized State Representations), an attack-resilient, model-free RL controller that enables robust UAV operation under adversarial sensor manipulation. Instead of relying on raw sensor observations, ARMOR learns a robust latent representation of the UAV's physical state via a two-stage training framework. In the first stage, a teacher encoder, trained with privileged attack information, generates attack-aware latent states for RL policy training. In the second stage, a student encoder is trained via supervised learning to approximate the teacher's latent states using only historical sensor data, enabling real-world deployment without privileged information. Our experiments show that ARMOR outperforms conventional methods, ensuring UAV safety. Additionally, ARMOR improves generalization to unseen attacks and reduces training cost by eliminating the need for iterative adversarial training.