Learning a Structured Neural Network Policy for a Hopping Task

TL;DR

This paper introduces a neural network-based reactive policy for dynamic hopping tasks with unknown contact points, leveraging optimal control and a novel architecture to improve robustness and interpretability, demonstrated through simulations and real robot transfer.

Contribution

It presents a new method combining optimal control and neural networks with a novel architecture for reactive locomotion policies in contact-rich tasks.

Findings

Outperforms model-free gradient methods in simulation.

Demonstrates robustness to environmental changes.

Successfully transfers policies to real robots.

Abstract

In this work we present a method for learning a reactive policy for a simple dynamic locomotion task involving hard impact and switching contacts where we assume the contact location and contact timing to be unknown. To learn such a policy, we use optimal control to optimize a local controller for a fixed environment and contacts. We learn the contact-rich dynamics for our underactuated systems along these trajectories in a sample efficient manner. We use the optimized policies to learn the reactive policy in form of a neural network. Using a new neural network architecture, we are able to preserve more information from the local policy and make its output interpretable in the sense that its output in terms of desired trajectories, feedforward commands and gains can be interpreted. Extensive simulations demonstrate the robustness of the approach to changing environments, outperforming a model-free gradient policy based methods on the same tasks in simulation. Finally, we show that the learned policy can be robustly transferred on a real robot.