Precision matters: Precision-aware ensemble for weakly supervised semantic segmentation

TL;DR

This paper introduces ORANDNet, a precision-aware ensemble method for weakly supervised semantic segmentation that combines CAMs from different classifiers and uses curriculum learning to improve pseudo-mask quality and segmentation accuracy.

Contribution

The paper proposes ORANDNet, an ensemble approach that enhances WSSS by increasing pseudo-mask precision through classifier combination and curriculum learning.

Findings

ORANDNet improves segmentation performance over single models.

Combining CAMs from different classifiers yields better results.

Curriculum learning helps reduce noise in pseudo-masks.

Abstract

Weakly Supervised Semantic Segmentation (WSSS) employs weak supervision, such as image-level labels, to train the segmentation model. Despite the impressive achievement in recent WSSS methods, we identify that introducing weak labels with high mean Intersection of Union (mIoU) does not guarantee high segmentation performance. Existing studies have emphasized the importance of prioritizing precision and reducing noise to improve overall performance. In the same vein, we propose ORANDNet, an advanced ensemble approach tailored for WSSS. ORANDNet combines Class Activation Maps (CAMs) from two different classifiers to increase the precision of pseudo-masks (PMs). To further mitigate small noise in the PMs, we incorporate curriculum learning. This involves training the segmentation model initially with pairs of smaller-sized images and corresponding PMs, gradually transitioning to the original-sized pairs. By combining the original CAMs of ResNet-50 and ViT, we significantly improve the segmentation performance over the single-best model and the naive ensemble model, respectively. We further extend our ensemble method to CAMs from AMN (ResNet-like) and MCTformer (ViT-like) models, achieving performance benefits in advanced WSSS models. It highlights the potential of our ORANDNet as a final add-on module for WSSS models.