Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamics

TL;DR

This paper critically evaluates zero-shot sim-to-real transfer methods for complex dynamics, revealing that simple random force injection rivals more complex approaches in real-world tasks.

Contribution

The study provides a thorough evaluation of various transfer methods, highlighting the surprising effectiveness of simple random force injection over complex techniques.

Findings

Random force injection performs as well as complex methods.

Complex methods require significant engineering and fine-tuning.

Simple approaches can be surprisingly effective in real-world transfer.

Abstract

Zero-shot sim-to-real transfer of tasks with complex dynamics is a highly challenging and unsolved problem. A number of solutions have been proposed in recent years, but we have found that many works do not present a thorough evaluation in the real world, or underplay the significant engineering effort and task-specific fine tuning that is required to achieve the published results. In this paper, we dive deeper into the sim-to-real transfer challenge, investigate why this is such a difficult problem, and present objective evaluations of a number of transfer methods across a range of real-world tasks. Surprisingly, we found that a method which simply injects random forces into the simulation performs just as well as more complex methods, such as those which randomise the simulator's dynamics parameters, or adapt a policy online using recurrent network architectures.