Benchmarking Domain Randomisation for Visual Sim-to-Real Transfer

TL;DR

This paper conducts a comprehensive benchmarking of domain randomisation techniques for visual sim-to-real transfer in robotics, highlighting the importance of rendering quality and the types of randomisation for effective transfer.

Contribution

It provides a detailed analysis of design choices in domain randomisation, including rendering quality and randomisation types, with empirical validation on a real-world object pose estimation task.

Findings

High-quality images outperform low-quality images in transfer.

Both distractors and textures are crucial for generalisation.

A small number of high-quality images is more effective than many low-quality ones.

Abstract

Domain randomisation is a very popular method for visual sim-to-real transfer in robotics, due to its simplicity and ability to achieve transfer without any real-world images at all. Nonetheless, a number of design choices must be made to achieve optimal transfer. In this paper, we perform a comprehensive benchmarking study on these different choices, with two key experiments evaluated on a real-world object pose estimation task. First, we study the rendering quality, and find that a small number of high-quality images is superior to a large number of low-quality images. Second, we study the type of randomisation, and find that both distractors and textures are important for generalisation to novel environments.