Dual-Augmented Transformer Network for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces a dual-augmented transformer network combining CNN and ViT for weakly supervised semantic segmentation, effectively addressing incomplete object region detection and outperforming state-of-the-art methods on PASCAL VOC 2012.

Contribution

It proposes a novel dual network architecture with self-regularization that leverages both CNN and transformer models for improved WSSS performance.

Findings

Outperforms previous state-of-the-art on PASCAL VOC 2012

Demonstrates the effectiveness of dual-augmented learning

Shows significant improvement in object region discovery

Abstract

Weakly supervised semantic segmentation (WSSS), a fundamental computer vision task, which aims to segment out the object within only class-level labels. The traditional methods adopt the CNN-based network and utilize the class activation map (CAM) strategy to discover the object regions. However, such methods only focus on the most discriminative region of the object, resulting in incomplete segmentation. An alternative is to explore vision transformers (ViT) to encode the image to acquire the global semantic information. Yet, the lack of transductive bias to objects is a flaw of ViT. In this paper, we explore the dual-augmented transformer network with self-regularization constraints for WSSS. Specifically, we propose a dual network with both CNN-based and transformer networks for mutually complementary learning, where both networks augment the final output for enhancement. Massive systemic evaluations on the challenging PASCAL VOC 2012 benchmark demonstrate the effectiveness of our method, outperforming previous state-of-the-art methods.