Reinforcement Learning with Adaptive Curriculum Dynamics Randomization for Fault-Tolerant Robot Control

TL;DR

This paper introduces an adaptive curriculum reinforcement learning algorithm with dynamics randomization that enables quadruped robots to learn fault-tolerant control policies without additional failure detection modules, outperforming conventional methods.

Contribution

The paper presents a novel ACDR algorithm that adaptively trains robots for fault tolerance under actuator failure conditions using a hard2easy curriculum approach.

Findings

ACDR outperforms conventional algorithms in reward and walking distance.

Hard2easy curriculum is more effective than easy2hard for locomotion.

Robots trained with ACDR do not need failure detection modules.

Abstract

This study is aimed at addressing the problem of fault tolerance of quadruped robots to actuator failure, which is critical for robots operating in remote or extreme environments. In particular, an adaptive curriculum reinforcement learning algorithm with dynamics randomization (ACDR) is established. The ACDR algorithm can adaptively train a quadruped robot in random actuator failure conditions and formulate a single robust policy for fault-tolerant robot control. It is noted that the hard2easy curriculum is more effective than the easy2hard curriculum for quadruped robot locomotion. The ACDR algorithm can be used to build a robot system that does not require additional modules for detecting actuator failures and switching policies. Experimental results show that the ACDR algorithm outperforms conventional algorithms in terms of the average reward and walking distance.