MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training

TL;DR

This paper introduces a novel method for training pedestrian detectors on unannotated videos by augmenting images with synthetically generated human agents, improving detection accuracy without manual annotations.

Contribution

The paper presents a new approach that automatically generates annotations using synthetic humans and a Spawn Probability Map, enabling effective training on unannotated datasets.

Findings

Improves average precision by 5-13% over existing detectors.

Automatically extracts vanishing point and pedestrian scales from unannotated images.

Uses synthetic human-agents to augment data for training pedestrian detectors.

Abstract

We present a new method for training pedestrian detectors on an unannotated set of images. We produce a mixed reality dataset that is composed of real-world background images and synthetically generated static human-agents. Our approach is general, robust, and makes no other assumptions about the unannotated dataset regarding the number or location of pedestrians. We automatically extract from the dataset: i) the vanishing point to calibrate the virtual camera, and ii) the pedestrians' scales to generate a Spawn Probability Map, which is a novel concept that guides our algorithm to place the pedestrians at appropriate locations. After putting synthetic human-agents in the unannotated images, we use these augmented images to train a Pedestrian Detector, with the annotations generated along with the synthetic agents. We conducted our experiments using Faster R-CNN by comparing the detection results on the unannotated dataset performed by the detector trained using our approach and detectors trained with other manually labeled datasets. We showed that our approach improves the average precision by 5-13% over these detectors.