Unlimited Road-scene Synthetic Annotation (URSA) Dataset

TL;DR

This paper introduces URSA, a large-scale synthetic road-scene dataset generated from sandbox video game engines, enabling efficient, on-demand ground truth annotations to improve semantic segmentation training.

Contribution

The paper presents a novel method for persistent, ground truth annotation of synthetic road scenes using open-source game modding tools, creating a large-scale dataset for deep learning.

Findings

Synthetic data improves segmentation model performance.

Over 1 million images collected for dataset.

Qualitative and quantitative gains over previous game-based datasets.

Abstract

In training deep neural networks for semantic segmentation, the main limiting factor is the low amount of ground truth annotation data that is available in currently existing datasets. The limited availability of such data is due to the time cost and human effort required to accurately and consistently label real images on a pixel level. Modern sandbox video game engines provide open world environments where traffic and pedestrians behave in a pseudo-realistic manner. This caters well to the collection of a believable road-scene dataset. Utilizing open-source tools and resources found in single-player modding communities, we provide a method for persistent, ground truth, asset annotation of a game world. By collecting a synthetic dataset containing upwards of $1,000,000$ images, we demonstrate real-time, on-demand, ground truth data annotation capability of our method. Supplementing this synthetic data to Cityscapes dataset, we show that our data generation method provides qualitative as well as quantitative improvements---for training networks---over previous methods that use video games as surrogate.