Beyond Discriminative Regions: Saliency Maps as Alternatives to CAMs for Weakly Supervised Semantic Segmentation

TL;DR

This paper compares saliency maps and CAMs for weakly supervised semantic segmentation, showing saliencies can address CAMs' limitations and proposing a stochastic aggregation method to enhance saliency performance.

Contribution

It offers a comprehensive comparison of saliency maps and CAMs, introduces new evaluation metrics, and proposes a stochastic cropping technique to improve saliency-based segmentation.

Findings

Saliency maps effectively address CAMs' non-discriminative region issue.

New metrics provide a thorough assessment of WS3 methods.

Stochastic cropping enhances saliency performance significantly.

Abstract

In recent years, several Weakly Supervised Semantic Segmentation (WS3) methods have been proposed that use class activation maps (CAMs) generated by a classifier to produce pseudo-ground truths for training segmentation models. While CAMs are good at highlighting discriminative regions (DR) of an image, they are known to disregard regions of the object that do not contribute to the classifier's prediction, termed non-discriminative regions (NDR). In contrast, attribution methods such as saliency maps provide an alternative approach for assigning a score to every pixel based on its contribution to the classification prediction. This paper provides a comprehensive comparison between saliencies and CAMs for WS3. Our study includes multiple perspectives on understanding their similarities and dissimilarities. Moreover, we provide new evaluation metrics that perform a comprehensive assessment of WS3 performance of alternative methods w.r.t. CAMs. We demonstrate the effectiveness of saliencies in addressing the limitation of CAMs through our empirical studies on benchmark datasets. Furthermore, we propose random cropping as a stochastic aggregation technique that improves the performance of saliency, making it a strong alternative to CAM for WS3.