Exploiting saliency for object segmentation from image level labels

TL;DR

This paper introduces a method that leverages saliency models alongside image-level labels to improve weakly supervised object segmentation, achieving near fully-supervised performance without pixel-level annotations.

Contribution

It proposes combining saliency models with image-level labels to better estimate object extents, advancing weakly supervised semantic segmentation.

Findings

Achieved 80% of fully supervised performance

Outperformed previous weakly supervised methods

Demonstrated effectiveness of saliency integration

Abstract

There have been remarkable improvements in the semantic labelling task in the recent years. However, the state of the art methods rely on large-scale pixel-level annotations. This paper studies the problem of training a pixel-wise semantic labeller network from image-level annotations of the present object classes. Recently, it has been shown that high quality seeds indicating discriminative object regions can be obtained from image-level labels. Without additional information, obtaining the full extent of the object is an inherently ill-posed problem due to co-occurrences. We propose using a saliency model as additional information and hereby exploit prior knowledge on the object extent and image statistics. We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling. The code is available at https://goo.gl/KygSeb.