Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features

TL;DR

This paper introduces an iterative framework for weakly-supervised semantic segmentation that progressively refines object regions by mining common features and using saliency maps, leading to improved accuracy.

Contribution

The novel iterative bottom-up and top-down approach effectively expands object regions and refines segmentation using mined features and saliency maps, outperforming previous methods.

Findings

Outperforms state-of-the-art on Pascal VOC 2012

Progressively refines object masks through iterations

Effectively combines feature mining and saliency for segmentation

Abstract

Weakly-supervised semantic segmentation under image tags supervision is a challenging task as it directly associates high-level semantic to low-level appearance. To bridge this gap, in this paper, we propose an iterative bottom-up and top-down framework which alternatively expands object regions and optimizes segmentation network. We start from initial localization produced by classification networks. While classification networks are only responsive to small and coarse discriminative object regions, we argue that, these regions contain significant common features about objects. So in the bottom-up step, we mine common object features from the initial localization and expand object regions with the mined features. To supplement non-discriminative regions, saliency maps are then considered under Bayesian framework to refine the object regions. Then in the top-down step, the refined object regions are used as supervision to train the segmentation network and to predict object masks. These object masks provide more accurate localization and contain more regions of object. Further, we take these object masks as initial localization and mine common object features from them. These processes are conducted iteratively to progressively produce fine object masks and optimize segmentation networks. Experimental results on Pascal VOC 2012 dataset demonstrate that the proposed method outperforms previous state-of-the-art methods by a large margin.