Pseudo-mask Matters in Weakly-supervised Semantic Segmentation

TL;DR

This paper introduces novel techniques for generating high-quality pseudo-masks in weakly-supervised semantic segmentation, significantly improving performance by addressing noise and quality issues in pseudo-mask creation and training.

Contribution

The paper proposes four innovative methods—Coefficient of Variation Smoothing, Proportional Pseudo-mask Generation, Pretended Under-Fitting, and Cyclic Pseudo-mask—to enhance pseudo-mask quality and training robustness in WSSS.

Findings

Achieved state-of-the-art mIoU of 70.0% on Pascal VOC 2012

Achieved state-of-the-art mIoU of 40.2% on MS COCO 2014

Demonstrated effectiveness of proposed methods through extensive experiments.

Abstract

Most weakly supervised semantic segmentation (WSSS) methods follow the pipeline that generates pseudo-masks initially and trains the segmentation model with the pseudo-masks in fully supervised manner after. However, we find some matters related to the pseudo-masks, including high quality pseudo-masks generation from class activation maps (CAMs), and training with noisy pseudo-mask supervision. For these matters, we propose the following designs to push the performance to new state-of-art: (i) Coefficient of Variation Smoothing to smooth the CAMs adaptively; (ii) Proportional Pseudo-mask Generation to project the expanded CAMs to pseudo-mask based on a new metric indicating the importance of each class on each location, instead of the scores trained from binary classifiers. (iii) Pretended Under-Fitting strategy to suppress the influence of noise in pseudo-mask; (iv) Cyclic Pseudo-mask to boost the pseudo-masks during training of fully supervised semantic segmentation (FSSS). Experiments based on our methods achieve new state-of-art results on two changeling weakly supervised semantic segmentation datasets, pushing the mIoU to 70.0% and 40.2% on PAS-CAL VOC 2012 and MS COCO 2014 respectively. Codes including segmentation framework are released at https://github.com/Eli-YiLi/PMM