Robust Deep Reinforcement Learning for Quadcopter Control

TL;DR

This paper introduces a robust deep reinforcement learning approach for quadcopter control that enhances policy transferability across varying environments by integrating Robust Markov Decision Processes, leading to improved generalization and adaptability.

Contribution

It proposes a novel robust RL method using RMDP for drone control, improving transferability and robustness over standard RL policies.

Findings

Robust policies outperform standard agents in unseen environments.

Increased robustness enhances generalization to non-stationary environments.

Method demonstrates effective transfer from simulation to varied test conditions.

Abstract

Deep reinforcement learning (RL) has made it possible to solve complex robotics problems using neural networks as function approximators. However, the policies trained on stationary environments suffer in terms of generalization when transferred from one environment to another. In this work, we use Robust Markov Decision Processes (RMDP) to train the drone control policy, which combines ideas from Robust Control and RL. It opts for pessimistic optimization to handle potential gaps between policy transfer from one environment to another. The trained control policy is tested on the task of quadcopter positional control. RL agents were trained in a MuJoCo simulator. During testing, different environment parameters (unseen during the training) were used to validate the robustness of the trained policy for transfer from one environment to another. The robust policy outperformed the standard agents in these environments, suggesting that the added robustness increases generality and can adapt to non-stationary environments. Codes: https://github.com/adipandas/gym_multirotor