One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model

TL;DR

This paper introduces a simple prediction filtering technique for semi-weakly supervised semantic segmentation that improves existing methods by leveraging classifier confidence to ignore unlikely classes, leading to better performance.

Contribution

The paper proposes a novel, simple post-processing method called prediction filtering that enhances semi-weakly supervised segmentation models without complex pseudo-label extraction.

Findings

Prediction filtering improves baseline SWSSS performance.

Combining prediction filtering with existing methods yields better results.

The approach is simple, effective, and compatible with other SWSSS techniques.

Abstract

Semi-weakly supervised semantic segmentation (SWSSS) aims to train a model to identify objects in images based on a small number of images with pixel-level labels, and many more images with only image-level labels. Most existing SWSSS algorithms extract pixel-level pseudo-labels from an image classifier - a very difficult task to do well, hence requiring complicated architectures and extensive hyperparameter tuning on fully-supervised validation sets. We propose a method called prediction filtering, which instead of extracting pseudo-labels, just uses the classifier as a classifier: it ignores any segmentation predictions from classes which the classifier is confident are not present. Adding this simple post-processing method to baselines gives results competitive with or better than prior SWSSS algorithms. Moreover, it is compatible with pseudo-label methods: adding prediction filtering to existing SWSSS algorithms further improves segmentation performance.