Soft Self-labeling and Potts Relaxations for Weakly-Supervised Segmentation

TL;DR

This paper introduces a soft self-labeling approach with Potts relaxations for weakly-supervised segmentation, improving training and performance over existing methods by effectively handling uncertainty and errors in pseudo-labels.

Contribution

It proposes a principled auxiliary loss and a continuous sub-problem solver, enhancing weakly-supervised segmentation with soft pseudo-labels and systematic evaluation of CRF relaxations.

Findings

Soft self-labeling improves scribble-based training.

Outperforms complex WSSS systems and even full supervision.

Systematic evaluation of CRF relaxations and neighborhood systems.

Abstract

We consider weakly supervised segmentation where only a fraction of pixels have ground truth labels (scribbles) and focus on a self-labeling approach optimizing relaxations of the standard unsupervised CRF/Potts loss on unlabeled pixels. While WSSS methods can directly optimize such losses via gradient descent, prior work suggests that higher-order optimization can improve network training by introducing hidden pseudo-labels and powerful CRF sub-problem solvers, e.g. graph cut. However, previously used hard pseudo-labels can not represent class uncertainty or errors, which motivates soft self-labeling. We derive a principled auxiliary loss and systematically evaluate standard and new CRF relaxations (convex and non-convex), neighborhood systems, and terms connecting network predictions with soft pseudo-labels. We also propose a general continuous sub-problem solver. Using only standard architectures, soft self-labeling consistently improves scribble-based training and outperforms significantly more complex specialized WSSS systems. It can outperform full pixel-precise supervision. Our general ideas apply to other weakly-supervised problems/systems.