Learning natural locomotion behaviors for humanoid robots using human knowledge

TL;DR

This paper introduces a novel learning framework combining imitation learning, reinforcement learning, and control theories to enable humanoid robots to perform natural, dynamic, and robust human-like locomotion.

Contribution

The paper proposes a new framework that integrates human motion data and a Multi-Expert network to improve humanoid locomotion learning, with an explainable reward design and rigorous validation.

Findings

Produced robust locomotion behaviors across various scenarios

Demonstrated resilience to terrain irregularities and external pushes

Validated framework's effectiveness through benchmarking

Abstract

This paper presents a new learning framework that leverages the knowledge from imitation learning, deep reinforcement learning, and control theories to achieve human-style locomotion that is natural, dynamic, and robust for humanoids. We proposed novel approaches to introduce human bias, i.e. motion capture data and a special Multi-Expert network structure. We used the Multi-Expert network structure to smoothly blend behavioral features, and used the augmented reward design for the task and imitation rewards. Our reward design is composable, tunable, and explainable by using fundamental concepts from conventional humanoid control. We rigorously validated and benchmarked the learning framework which consistently produced robust locomotion behaviors in various test scenarios. Further, we demonstrated the capability of learning robust and versatile policies in the presence of disturbances, such as terrain irregularities and external pushes.