Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces AdvCAM, an anti-adversarial attribution method that enhances weakly and semi-supervised semantic segmentation by progressively identifying more object regions, achieving state-of-the-art results on PASCAL VOC 2012.

Contribution

It proposes a novel anti-adversarial manipulation of attribution maps and a regularization technique to improve segmentation accuracy.

Findings

Achieves 68.0 mIoU in weakly supervised segmentation.

Achieves 76.9 mIoU in semi-supervised segmentation.

Sets new state-of-the-art performance on PASCAL VOC 2012.

Abstract

Weakly supervised semantic segmentation produces a pixel-level localization from a classifier, but it is likely to restrict its focus to a small discriminative region of the target object. AdvCAM is an attribution map of an image that is manipulated to increase the classification score. This manipulation is realized in an anti-adversarial manner, which perturbs the images along pixel gradients in the opposite direction from those used in an adversarial attack. It forces regions initially considered not to be discriminative to become involved in subsequent classifications, and produces attribution maps that successively identify more regions of the target object. In addition, we introduce a new regularization procedure that inhibits the incorrect attribution of regions unrelated to the target object and limits the attributions of the regions that already have high scores. On PASCAL VOC 2012 test images, we achieve mIoUs of 68.0 and 76.9 for weakly and semi-supervised semantic segmentation respectively, which represent a new state-of-the-art.