Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks

TL;DR

This paper introduces a benchmark setup for evaluating randomization strategies in Sim2Real transfer for robotic manipulation, revealing that full randomization and fine-tuning improve real-world transfer but may hinder simulation policy learning.

Contribution

It provides a standardized, reproducible experimental setup for comparing randomization techniques in robotic Sim2Real transfer.

Findings

More randomization improves Sim2Real transfer.

Full randomization and fine-tuning outperform other methods.

Excessive randomization can hinder policy learning in simulation.

Abstract

Randomization is currently a widely used approach in Sim2Real transfer for data-driven learning algorithms in robotics. Still, most Sim2Real studies report results for a specific randomization technique and often on a highly customized robotic system, making it difficult to evaluate different randomization approaches systematically. To address this problem, we define an easy-to-reproduce experimental setup for a robotic reach-and-balance manipulator task, which can serve as a benchmark for comparison. We compare four randomization strategies with three randomized parameters both in simulation and on a real robot. Our results show that more randomization helps in Sim2Real transfer, yet it can also harm the ability of the algorithm to find a good policy in simulation. Fully randomized simulations and fine-tuning show differentiated results and translate better to the real robot than the other approaches tested.