Weakly- and Semi-Supervised Object Detection with Expectation-Maximization Algorithm

TL;DR

This paper introduces an EM-based deep learning approach for object detection using only image-level labels, significantly improving performance in weakly- and semi-supervised settings and approaching fully supervised results.

Contribution

It develops a novel EM algorithm for CNN-based object detection applicable to weakly- and semi-supervised learning, outperforming existing methods on benchmark datasets.

Findings

Significant performance improvement over state-of-the-art in weakly supervised detection.

Approaches the accuracy of fully supervised Fast R-CNN with few strongly labeled images.

Demonstrates effectiveness on PASCAL VOC 2007 dataset.

Abstract

Object detection when provided image-level labels instead of instance-level labels (i.e., bounding boxes) during training is an important problem in computer vision, since large scale image datasets with instance-level labels are extremely costly to obtain. In this paper, we address this challenging problem by developing an Expectation-Maximization (EM) based object detection method using deep convolutional neural networks (CNNs). Our method is applicable to both the weakly-supervised and semi-supervised settings. Extensive experiments on PASCAL VOC 2007 benchmark show that (1) in the weakly supervised setting, our method provides significant detection performance improvement over current state-of-the-art methods, (2) having access to a small number of strongly (instance-level) annotated images, our method can almost match the performace of the fully supervised Fast RCNN. We share our source code at https://github.com/ZiangYan/EM-WSD.