Learning structure-aware semantic segmentation with image-level supervision

TL;DR

This paper enhances weakly-supervised semantic segmentation by incorporating structure-aware techniques, including boundary detection and smoothness constraints, to improve the quality of class activation maps and segmentation accuracy.

Contribution

It introduces an auxiliary boundary detection module and smoothness loss to preserve structure information in weakly-supervised segmentation, addressing limitations of traditional CAM-based methods.

Findings

Improved segmentation accuracy on PASCAL-VOC dataset

Enhanced boundary sharpness and consistency in predictions

Demonstrated effectiveness of structure-aware supervision

Abstract

Compared with expensive pixel-wise annotations, image-level labels make it possible to learn semantic segmentation in a weakly-supervised manner. Within this pipeline, the class activation map (CAM) is obtained and further processed to serve as a pseudo label to train the semantic segmentation model in a fully-supervised manner. In this paper, we argue that the lost structure information in CAM limits its application in downstream semantic segmentation, leading to deteriorated predictions. Furthermore, the inconsistent class activation scores inside the same object contradicts the common sense that each region of the same object should belong to the same semantic category. To produce sharp prediction with structure information, we introduce an auxiliary semantic boundary detection module, which penalizes the deteriorated predictions. Furthermore, we adopt smoothness loss to encourage prediction inside the object to be consistent. Experimental results on the PASCAL-VOC dataset illustrate the effectiveness of the proposed solution.