Weakly Supervised Semantic Segmentation using Out-of-Distribution Data

TL;DR

This paper introduces a novel approach called W-OoD that leverages out-of-distribution data to improve weakly supervised semantic segmentation by reducing spurious background cues, achieving state-of-the-art results.

Contribution

The paper proposes using hard out-of-distribution data to distinguish foreground from background, enhancing weakly supervised segmentation without extensive additional annotation.

Findings

W-OoD outperforms previous methods on Pascal VOC 2012.

Utilizing hard OoD data effectively suppresses spurious background cues.

The approach requires minimal extra annotation effort.

Abstract

Weakly supervised semantic segmentation (WSSS) methods are often built on pixel-level localization maps obtained from a classifier. However, training on class labels only, classifiers suffer from the spurious correlation between foreground and background cues (e.g. train and rail), fundamentally bounding the performance of WSSS. There have been previous endeavors to address this issue with additional supervision. We propose a novel source of information to distinguish foreground from the background: Out-of-Distribution (OoD) data, or images devoid of foreground object classes. In particular, we utilize the hard OoDs that the classifier is likely to make false-positive predictions. These samples typically carry key visual features on the background (e.g. rail) that the classifiers often confuse as foreground (e.g. train), so these cues let classifiers correctly suppress spurious background cues. Acquiring such hard OoDs does not require an extensive amount of annotation efforts; it only incurs a few additional image-level labeling costs on top of the original efforts to collect class labels. We propose a method, W-OoD, for utilizing the hard OoDs. W-OoD achieves state-of-the-art performance on Pascal VOC 2012.