Adaptive Mimic: Deep Reinforcement Learning of Parameterized Bipedal Walking from Infeasible References

TL;DR

This paper introduces an adaptive reinforcement learning approach for bipedal walking that effectively learns from low-quality or infeasible references, improving robustness and performance beyond traditional reference-based methods.

Contribution

It proposes an adaptive reward function that allows imitation learning from low-quality references, enabling more flexible and efficient bipedal locomotion learning.

Findings

Adaptive reward function improves learning efficiency.

Low-quality references can replace high-quality ones.

Method achieves robust bipedal walking performance.

Abstract

Not until recently, robust robot locomotion has been achieved by deep reinforcement learning (DRL). However, for efficient learning of parametrized bipedal walking, developed references are usually required, limiting the performance to that of the references. In this paper, we propose to design an adaptive reward function for imitation learning from the references. The agent is encouraged to mimic the references when its performance is low, while to pursue high performance when it reaches the limit of references. We further demonstrate that developed references can be replaced by low-quality references that are generated without laborious tuning and infeasible to deploy by themselves, as long as they can provide a priori knowledge to expedite the learning process.