Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

TL;DR

This paper introduces EM-based methods for training deep convolutional neural networks for semantic image segmentation using weakly or semi-supervised data, achieving competitive results with less annotation effort.

Contribution

It presents novel EM algorithms for weakly and semi-supervised semantic segmentation, reducing annotation requirements while maintaining high performance.

Findings

Achieved competitive results on PASCAL VOC 2012

Reduced annotation effort significantly

Demonstrated effectiveness of EM-based training methods

Abstract

Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation. We study the more challenging problem of learning DCNNs for semantic image segmentation from either (1) weakly annotated training data such as bounding boxes or image-level labels or (2) a combination of few strongly labeled and many weakly labeled images, sourced from one or multiple datasets. We develop Expectation-Maximization (EM) methods for semantic image segmentation model training under these weakly supervised and semi-supervised settings. Extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentation benchmark, while requiring significantly less annotation effort. We share source code implementing the proposed system at https://bitbucket.org/deeplab/deeplab-public.