MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?

TL;DR

This paper introduces MOTSynth, a large synthetic dataset created with a rendering engine, demonstrating its effectiveness as a substitute for real data in pedestrian detection and tracking tasks, addressing privacy and annotation challenges.

Contribution

MOTSynth is a novel, diverse synthetic dataset that can replace real data for training in pedestrian detection and tracking, reducing privacy and annotation issues.

Findings

MOTSynth achieves comparable performance to real data in detection tasks.

Synthetic data improves privacy and reduces annotation effort.

Experiments validate MOTSynth's effectiveness across multiple tasks.

Abstract

Deep learning-based methods for video pedestrian detection and tracking require large volumes of training data to achieve good performance. However, data acquisition in crowded public environments raises data privacy concerns -- we are not allowed to simply record and store data without the explicit consent of all participants. Furthermore, the annotation of such data for computer vision applications usually requires a substantial amount of manual effort, especially in the video domain. Labeling instances of pedestrians in highly crowded scenarios can be challenging even for human annotators and may introduce errors in the training data. In this paper, we study how we can advance different aspects of multi-person tracking using solely synthetic data. To this end, we generate MOTSynth, a large, highly diverse synthetic dataset for object detection and tracking using a rendering game engine. Our experiments show that MOTSynth can be used as a replacement for real data on tasks such as pedestrian detection, re-identification, segmentation, and tracking.