Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion

TL;DR

This paper demonstrates that incorporating passive body elements in biped robots via model-based reinforcement learning leads to robust, energy-efficient locomotion by leveraging stable limit cycles from body-ground interactions.

Contribution

It shows that passive body dynamics can be exploited in reinforcement learning to achieve high-performance, energy-efficient bipedal locomotion, highlighting the importance of passive properties in embodied AI.

Findings

Passive elements lead to robust, energy-efficient locomotion.

Attractor-driven learning enables convergence to stable limit cycles.

Passive body models outperform non-passive models in locomotion tasks.

Abstract

Embodiment is a significant keyword in recent machine learning fields. This study focused on the passive nature of the body of a biped robot to generate walking and running locomotion using model-based deep reinforcement learning. We constructed two models in a simulator, one with passive elements (e.g., springs) and the other, which is similar to general humanoids, without passive elements. The training of the model with passive elements was highly affected by the attractor of the system. This lead that although the trajectories quickly converged to limit cycles, it took a long time to obtain large rewards. However, thanks to the attractor-driven learning, the acquired locomotion was robust and energy-efficient. The results revealed that robots with passive elements could efficiently acquire high-performance locomotion by utilizing stable limit cycles generated through dynamic interaction between the body and ground. This study demonstrates the importance of implementing passive properties in the body for future embodied AI.