Continuous Control Reinforcement Learning: Distributed Distributional DrQ Algorithms

TL;DR

This paper introduces Distributed Distributional DrQ, an advanced off-policy reinforcement learning algorithm for continuous control, enhancing performance through distributional value functions and distributed actor policies.

Contribution

It extends DrQ-v2 by integrating distributional critics and distributed actors, achieving improved results in challenging continuous control tasks.

Findings

Outperforms previous algorithms on several benchmarks

Enhanced expression ability of value functions

Better handling of high-dimensional control tasks

Abstract

Distributed Distributional DrQ is a model-free and off-policy RL algorithm for continuous control tasks based on the state and observation of the agent, which is an actor-critic method with the data-augmentation and the distributional perspective of critic value function. Aim to learn to control the agent and master some tasks in a high-dimensional continuous space. DrQ-v2 uses DDPG as the backbone and achieves out-performance in various continuous control tasks. Here Distributed Distributional DrQ uses Distributed Distributional DDPG as the backbone, and this modification aims to achieve better performance in some hard continuous control tasks through the better expression ability of distributional value function and distributed actor policies.