Unity Perception: Generate Synthetic Data for Computer Vision

TL;DR

Unity Perception is an open-source toolkit that simplifies creating customizable synthetic datasets for computer vision, demonstrating improved model training outcomes with synthetic data over real data.

Contribution

It introduces a flexible, easy-to-use Unity-based framework for generating annotated synthetic datasets with customizable randomization for computer vision tasks.

Findings

Synthetic data trained models outperform real data trained models

The package enables rapid dataset generation with annotations

Demonstrates effectiveness of synthetic data in training vision models

Abstract

We introduce the Unity Perception package which aims to simplify and accelerate the process of generating synthetic datasets for computer vision tasks by offering an easy-to-use and highly customizable toolset. This open-source package extends the Unity Editor and engine components to generate perfectly annotated examples for several common computer vision tasks. Additionally, it offers an extensible Randomization framework that lets the user quickly construct and configure randomized simulation parameters in order to introduce variation into the generated datasets. We provide an overview of the provided tools and how they work, and demonstrate the value of the generated synthetic datasets by training a 2D object detection model. The model trained with mostly synthetic data outperforms the model trained using only real data.