UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environments

TL;DR

UnrealROX+ enhances a virtual data generation tool by making it more flexible, customizable, and easier to integrate with existing Unreal projects, supporting diverse data types for computer vision tasks.

Contribution

It introduces UnrealROX+, a modular, user-friendly plugin that extends the original UnrealROX capabilities with new features and improved usability for synthetic data generation.

Findings

More flexible data generation workflows

Easier integration with existing Unreal projects

Support for new data types like albedo and Python API

Abstract

Synthetic data generation has become essential in last years for feeding data-driven algorithms, which surpassed traditional techniques performance in almost every computer vision problem. Gathering and labelling the amount of data needed for these data-hungry models in the real world may become unfeasible and error-prone, while synthetic data give us the possibility of generating huge amounts of data with pixel-perfect annotations. However, most synthetic datasets lack from enough realism in their rendered images. In that context UnrealROX generation tool was presented in 2019, allowing to generate highly realistic data, at high resolutions and framerates, with an efficient pipeline based on Unreal Engine, a cutting-edge videogame engine. UnrealROX enabled robotic vision researchers to generate realistic and visually plausible data with full ground truth for a wide variety of problems such as class and instance semantic segmentation, object detection, depth estimation, visual grasping, and navigation. Nevertheless, its workflow was very tied to generate image sequences from a robotic on-board camera, making hard to generate data for other purposes. In this work, we present UnrealROX+, an improved version of UnrealROX where its decoupled and easy-to-use data acquisition system allows to quickly design and generate data in a much more flexible and customizable way. Moreover, it is packaged as an Unreal plug-in, which makes it more comfortable to use with already existing Unreal projects, and it also includes new features such as generating albedo or a Python API for interacting with the virtual environment from Deep Learning frameworks.