Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces KTSE, a novel weakly supervised semantic segmentation method that uses simulated inter-image erasing and knowledge transfer to improve object localization, addressing over-activation and under-activation issues.

Contribution

The paper proposes a new simulated inter-image erasing technique combined with knowledge transfer and self-supervised regularization for better weakly supervised segmentation.

Findings

Outperforms existing methods on PASCAL VOC 2012 and COCO datasets.

Effectively balances object activation to improve segmentation accuracy.

Demonstrates the effectiveness of multi-granularity alignment and regularization modules.

Abstract

Though adversarial erasing has prevailed in weakly supervised semantic segmentation to help activate integral object regions, existing approaches still suffer from the dilemma of under-activation and over-expansion due to the difficulty in determining when to stop erasing. In this paper, we propose a \textbf{K}nowledge \textbf{T}ransfer with \textbf{S}imulated Inter-Image \textbf{E}rasing (KTSE) approach for weakly supervised semantic segmentation to alleviate the above problem. In contrast to existing erasing-based methods that remove the discriminative part for more object discovery, we propose a simulated inter-image erasing scenario to weaken the original activation by introducing extra object information. Then, object knowledge is transferred from the anchor image to the consequent less activated localization map to strengthen network localization ability. Considering the adopted bidirectional alignment will also weaken the anchor image activation if appropriate constraints are missing, we propose a self-supervised regularization module to maintain the reliable activation in discriminative regions and improve the inter-class object boundary recognition for complex images with multiple categories of objects. In addition, we resort to intra-image erasing and propose a multi-granularity alignment module to gently enlarge the object activation to boost the object knowledge transfer. Extensive experiments and ablation studies on PASCAL VOC 2012 and COCO datasets demonstrate the superiority of our proposed approach. Source codes and models are available at https://github.com/NUST-Machine-Intelligence-Laboratory/KTSE.