Emergence of Human-comparable Balancing Behaviors by Deep Reinforcement Learning

TL;DR

This paper introduces a hierarchical deep reinforcement learning framework that enables humanoid robots to learn diverse, human-like balancing behaviors, surpassing traditional controllers by actively pushing off ankles and adhering to physical constraints.

Contribution

It presents a novel explainable reward design and demonstrates the emergence of human-like balancing behaviors in simulation using deep reinforcement learning.

Findings

Humanoid robots learned active push-off ankle behaviors.

The framework outperforms conventional zero moment point controllers.

Simulated results show human-like balance control behaviors.

Abstract

This paper presents a hierarchical framework based on deep reinforcement learning that learns a diversity of policies for humanoid balance control. Conventional zero moment point based controllers perform limited actions during under-actuation, whereas the proposed framework can perform human-like balancing behaviors such as active push-off of ankles. The learning is done through the design of an explainable reward based on physical constraints. The simulated results are presented and analyzed. The successful emergence of human-like behaviors through deep reinforcement learning proves the feasibility of using an AI-based approach for learning humanoid balancing control in a unified framework.