SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations

TL;DR

SSVOD introduces a semi-supervised video object detection framework that leverages motion dynamics and sparse annotations to improve detection accuracy while reducing annotation costs.

Contribution

The paper proposes a novel end-to-end semi-supervised framework that exploits temporal motion cues and introduces flow-warped predictions for robust pseudo-labeling in videos.

Findings

Significant performance improvements on ImageNet-VID, Epic-KITCHENS, and YouTube-VIS datasets.

Effective use of flow-warped predictions for temporal consistency.

Balanced pseudo-labeling approach reduces confirmation bias and noise.

Abstract

Despite significant progress in semi-supervised learning for image object detection, several key issues are yet to be addressed for video object detection: (1) Achieving good performance for supervised video object detection greatly depends on the availability of annotated frames. (2) Despite having large inter-frame correlations in a video, collecting annotations for a large number of frames per video is expensive, time-consuming, and often redundant. (3) Existing semi-supervised techniques on static images can hardly exploit the temporal motion dynamics inherently present in videos. In this paper, we introduce SSVOD, an end-to-end semi-supervised video object detection framework that exploits motion dynamics of videos to utilize large-scale unlabeled frames with sparse annotations. To selectively assemble robust pseudo-labels across groups of frames, we introduce \textit{flow-warped predictions} from nearby frames for temporal-consistency estimation. In particular, we introduce cross-IoU and cross-divergence based selection methods over a set of estimated predictions to include robust pseudo-labels for bounding boxes and class labels, respectively. To strike a balance between confirmation bias and uncertainty noise in pseudo-labels, we propose confidence threshold based combination of hard and soft pseudo-labels. Our method achieves significant performance improvements over existing methods on ImageNet-VID, Epic-KITCHENS, and YouTube-VIS datasets. Code and pre-trained models will be released.