Track and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection

TL;DR

This paper introduces a novel weakly-supervised object detection framework that leverages tracked object boxes from videos to automatically generate pseudo ground-truth boxes for training, reducing annotation costs.

Contribution

It proposes a new method to transfer tracked object boxes from videos to images, enabling automatic pseudo-label generation for weakly-supervised detection.

Findings

Achieves state-of-the-art results on PASCAL datasets

Effectively reduces reliance on manual bounding box annotations

Demonstrates robust transfer of video-tracked boxes to image detection

Abstract

The status quo approach to training object detectors requires expensive bounding box annotations. Our framework takes a markedly different direction: we transfer tracked object boxes from weakly-labeled videos to weakly-labeled images to automatically generate pseudo ground-truth boxes, which replace manually annotated bounding boxes. We first mine discriminative regions in the weakly-labeled image collection that frequently/rarely appear in the positive/negative images. We then match those regions to videos and retrieve the corresponding tracked object boxes. Finally, we design a hough transform algorithm to vote for the best box to serve as the pseudo GT for each image, and use them to train an object detector. Together, these lead to state-of-the-art weakly-supervised detection results on the PASCAL 2007 and 2010 datasets.