Dual Progressive Transformations for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces CRT, a novel CNN-refined transformer approach that improves weakly supervised semantic segmentation by generating more accurate class activation maps, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a CNN-refined transformer (CRT) that effectively combines global attention and local accuracy for WSSS, addressing over-activation issues of pure transformers.

Findings

CRT achieves new state-of-the-art performance on PASCAL VOC 2012.

CRT outperforms previous methods significantly in weakly supervised object localization.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

Weakly supervised semantic segmentation (WSSS), which aims to mine the object regions by merely using class-level labels, is a challenging task in computer vision. The current state-of-the-art CNN-based methods usually adopt Class-Activation-Maps (CAMs) to highlight the potential areas of the object, however, they may suffer from the part-activated issues. To this end, we try an early attempt to explore the global feature attention mechanism of vision transformer in WSSS task. However, since the transformer lacks the inductive bias as in CNN models, it can not boost the performance directly and may yield the over-activated problems. To tackle these drawbacks, we propose a Convolutional Neural Networks Refined Transformer (CRT) to mine a globally complete and locally accurate class activation maps in this paper. To validate the effectiveness of our proposed method, extensive experiments are conducted on PASCAL VOC 2012 and CUB-200-2011 datasets. Experimental evaluations show that our proposed CRT achieves the new state-of-the-art performance on both the weakly supervised semantic segmentation task the weakly supervised object localization task, which outperform others by a large margin.