Robot Learning from Randomized Simulations: A Review

TL;DR

This review discusses how domain randomization in randomized simulations helps robots learn control policies effectively, addressing the reality gap challenge in sim-to-real transfer for robotics.

Contribution

It provides a comprehensive overview of sim-to-real methods, focusing on domain randomization as a key technique for improving robot learning from simulations.

Findings

Domain randomization enhances transferability of learned policies.

Sim-to-real approaches reduce reliance on costly physical data.

The review highlights future directions in randomized simulation research.

Abstract

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the 'reality gap'. We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named 'domain randomization' which is a method for learning from randomized simulations.