Learning to Race in Minutes: Infoprop Dyna on the Mini Wheelbot

TL;DR

This paper demonstrates that Infoprop Dyna, a model-based reinforcement learning framework, enables a Mini Wheelbot to learn to race in just 11 minutes of real-world interaction without relying on simulators.

Contribution

It shows that Infoprop Dyna can directly learn effective control policies from real-world data for an underactuated robot, bypassing the need for simulators.

Findings

Mini Wheelbot learns to race in 11 minutes of real-world experience.

Infoprop Dyna successfully handles the robot's nonlinear, unstable dynamics.

The approach eliminates reliance on physics-based simulators for training.

Abstract

Reinforcement Learning (RL) has the potential to enable robots with fast, nonlinear, and unstable dynamics to reach the limits of their performance. However, most recent advances rely on carefully designed physics-based simulators and domain randomization to achieve successful sim-to-real transfer within reasonable wall-clock time. In this work, we bypass the need for such simulators and demonstrate that Infoprop Dyna, a state-of-the-art uncertainty-aware model-based reinforcement learning (MBRL) framework, can enable robots to learn directly from real-world interactions. Using Infoprop Dyna, the Mini Wheelbot, an underactuated unicycle robot, learns to race around a track within 11 minutes of real-world experience.