Rethinking Class Activation Maps for Segmentation: Revealing Semantic Information in Shallow Layers by Reducing Noise

TL;DR

This paper improves class activation maps for segmentation by analyzing shallow layers and introducing a gradient-based denoising method to reduce noise, leading to higher-quality semantic maps in weakly supervised learning.

Contribution

It reveals the semantic potential of shallow feature maps and proposes a simple gradient-based denoising technique to enhance CAM quality.

Findings

Shallow feature maps contain fine-grained semantic information with noise.

The proposed denoising method effectively reduces noise in CAMs.

Enhanced CAMs improve weakly supervised semantic segmentation performance.

Abstract

Class activation maps are widely used for explaining deep neural networks. Due to its ability to highlight regions of interest, it has evolved in recent years as a key step in weakly supervised learning. A major limitation to the performance of the class activation maps is the small spatial resolution of the feature maps in the last layer of the convolutional neural network. Therefore, we expect to generate high-resolution feature maps that result in high-quality semantic information. In this paper, we rethink the properties of semantic information in shallow feature maps. We find that the shallow feature maps still have fine-grained non-discriminative features while mixing considerable non-target noise. Furthermore, we propose a simple gradient-based denoising method to filter the noise by truncating the positive gradient. Our proposed scheme can be easily deployed in other CAM-related methods, facilitating these methods to obtain higher-quality class activation maps. We evaluate the proposed approach through a weakly-supervised semantic segmentation task, and a large number of experiments demonstrate the effectiveness of our approach.