Robo-PlaNet: Learning to Poke in a Day

TL;DR

Robo-PlaNet is an asynchronous extension of PlaNet that improves learning efficiency and performance in pixel-based robotic tasks by reducing training time and enabling faster data collection.

Contribution

We introduce Robo-PlaNet, an asynchronous variant of PlaNet that achieves higher performance more efficiently in robotic learning tasks.

Findings

Robo-PlaNet outperforms PlaNet in simulated environments.

Robo-PlaNet demonstrates superior results on real robotic experiments.

The asynchronous approach reduces training time significantly.

Abstract

Recently, the Deep Planning Network (PlaNet) approach was introduced as a model-based reinforcement learning method that learns environment dynamics directly from pixel observations. This architecture is useful for learning tasks in which either the agent does not have access to meaningful states (like position/velocity of robotic joints) or where the observed states significantly deviate from the physical state of the agent (which is commonly the case in low-cost robots in the form of backlash or noisy joint readings). PlaNet, by design, interleaves phases of training the dynamics model with phases of collecting more data on the target environment, leading to long training times. In this work, we introduce Robo-PlaNet, an asynchronous version of PlaNet. This algorithm consistently reaches higher performance in the same amount of time, which we demonstrate in both a simulated and a real robotic experiment.