Robust Deterministic Policy Gradient for Disturbance Attenuation and Its Application to Quadrotor Control

TL;DR

This paper introduces a robust reinforcement learning algorithm, RDPG, that formulates disturbance attenuation as a two-player game, leading to more resilient quadrotor control under severe disturbances.

Contribution

It develops a novel RDPG algorithm combining deterministic policy gradients with deep RL, and applies it to improve disturbance robustness in quadrotor control.

Findings

RDPG outperforms existing methods in disturbance attenuation.

RDDPG enhances stability and sample efficiency.

Experiments show superior robustness and tracking accuracy.

Abstract

This paper presents a robust reinforcement learning algorithm called robust deterministic policy gradient (RDPG), which reformulates the H-infinity control problem as a two-player zero-sum dynamic game between a user and an adversary. The method combines deterministic policy gradients with deep reinforcement learning to train a robust policy that attenuates disturbances efficiently. A practical variant, robust deep deterministic policy gradient (RDDPG), integrates twin-delayed updates for stability and sample efficiency. Experiments on an unmanned aerial vehicle demonstrate superior robustness and tracking accuracy under severe disturbance conditions.