Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces a novel weakly-supervised semantic segmentation framework that combines image-level labels with saliency maps to improve object boundary accuracy and reduce co-occurring pixel errors, achieving state-of-the-art results.

Contribution

The paper proposes Explicit Pseudo-pixel Supervision (EPS), a new method that leverages both localization and saliency maps for better pseudo-mask quality in WSSS.

Findings

Outperforms existing methods on PASCAL VOC 2012

Achieves new state-of-the-art on MS COCO 2014

Effectively improves object boundary accuracy

Abstract

Existing studies in weakly-supervised semantic segmentation (WSSS) using image-level weak supervision have several limitations: sparse object coverage, inaccurate object boundaries, and co-occurring pixels from non-target objects. To overcome these challenges, we propose a novel framework, namely Explicit Pseudo-pixel Supervision (EPS), which learns from pixel-level feedback by combining two weak supervisions; the image-level label provides the object identity via the localization map and the saliency map from the off-the-shelf saliency detection model offers rich boundaries. We devise a joint training strategy to fully utilize the complementary relationship between both information. Our method can obtain accurate object boundaries and discard co-occurring pixels, thereby significantly improving the quality of pseudo-masks. Experimental results show that the proposed method remarkably outperforms existing methods by resolving key challenges of WSSS and achieves the new state-of-the-art performance on both PASCAL VOC 2012 and MS COCO 2014 datasets.