# Data-Efficient and Safe Learning for Humanoid Locomotion Aided by a Dynamic Balancing Model

## TL;DR

This paper introduces a structured, data-efficient reinforcement learning approach for humanoid locomotion that combines a walking pattern generator, neural networks, and safety controllers to enhance safety and adaptability.

## Contribution

It proposes a novel structured control framework integrating a WPG, neural networks, and safety mechanisms for improved humanoid walking.

## Key findings

- Enhanced safety during learning through control-barrier functions
- Improved data efficiency using physics-based models
- Scalable approach applicable to various humanoid robots

## Abstract

In this letter, we formulate a novel Markov Decision Process (MDP) for safe and data-efficient learning for humanoid locomotion aided by a dynamic balancing model. In our previous studies of biped locomotion, we relied on a low-dimensional robot model, commonly used in high-level Walking Pattern Generators (WPGs). However, a low-level feedback controller cannot precisely track desired footstep locations due to the discrepancies between the full order model and the simplified model. In this study, we propose mitigating this problem by complementing a WPG with reinforcement learning. More specifically, we propose a structured footstep control method consisting of a WPG, a neural network, and a safety controller. The WPG provides an analytical method that promotes efficient learning while the neural network maximizes long-term rewards, and the safety controller encourages safe exploration based on step capturability and the use of control-barrier functions. Our contributions include the following (1) a structured learning control method for locomotion, (2) a data-efficient and safe learning process to improve walking using a physics-based model, and (3) the scalability of the procedure to various types of humanoid robots and walking.