Benchmarking Deep Reinforcement Learning for Navigation in Denied Sensor Environments

TL;DR

This paper benchmarks various deep reinforcement learning algorithms for autonomous navigation in environments with sensor denial, highlighting DreamerV3's superior performance and exploring adversarial training to enhance robustness.

Contribution

It introduces a comprehensive benchmark comparing DRL algorithms under sensor denial and evaluates adversarial training for robustness in navigation tasks.

Findings

DreamerV3 outperforms other DRL methods in visual navigation with dynamic goals.

Adversarial training improves robustness in sensor-denied environments.

Performance trade-offs exist when applying adversarial training to improve robustness.

Abstract

Deep Reinforcement learning (DRL) is used to enable autonomous navigation in unknown environments. Most research assume perfect sensor data, but real-world environments may contain natural and artificial sensor noise and denial. Here, we present a benchmark of both well-used and emerging DRL algorithms in a navigation task with configurable sensor denial effects. In particular, we are interested in comparing how different DRL methods (e.g. model-free PPO vs. model-based DreamerV3) are affected by sensor denial. We show that DreamerV3 outperforms other methods in the visual end-to-end navigation task with a dynamic goal - and other methods are not able to learn this. Furthermore, DreamerV3 generally outperforms other methods in sensor-denied environments. In order to improve robustness, we use adversarial training and demonstrate an improved performance in denied environments, although this generally comes with a performance cost on the vanilla environments. We anticipate this benchmark of different DRL methods and the usage of adversarial training to be a starting point for the development of more elaborate navigation strategies that are capable of dealing with uncertain and denied sensor readings.