ARM: A Confidence-Based Adversarial Reweighting Module for Coarse Semantic Segmentation

TL;DR

This paper introduces ARM, a confidence-based adversarial reweighting module that effectively leverages coarse annotations for semantic segmentation by distinguishing valuable pixels from mislabeled ones, improving performance on standard datasets.

Contribution

The paper proposes a novel confidence-based adversarial reweighting strategy, ARM, that simultaneously mines valuable pixels and suppresses mislabeled pixels, with proven convergence and improved segmentation results.

Findings

ARM improves mIoU on Cityscapes with coarse annotations

ARM achieves 47.50 mIoU on ADE20K dataset

The strategy enhances segmentation accuracy across datasets

Abstract

Coarsely-labeled semantic segmentation annotations are easy to obtain, but therefore bear the risk of losing edge details and introducing background pixels. Impeded by the inherent noise, existing coarse annotations are only taken as a bonus for model pre-training. In this paper, we try to exploit their potentials with a confidence-based reweighting strategy. To expand, loss-based reweighting strategies usually take the high loss value to identify two completely different types of pixels, namely, valuable pixels in noise-free annotations and mislabeled pixels in noisy annotations. This makes it impossible to perform two tasks of mining valuable pixels and suppressing mislabeled pixels at the same time. However, with the help of the prediction confidence, we successfully solve this dilemma and simultaneously perform two subtasks with a single reweighting strategy. Furthermore, we generalize this strategy into an Adversarial Reweighting Module (ARM) and prove its convergence strictly. Experiments on standard datasets shows our ARM can bring consistent improvements for both coarse annotations and fine annotations. Specifically, built on top of DeepLabv3+, ARM improves the mIoU on the coarsely-labeled Cityscapes by a considerable margin and increases the mIoU on the ADE20K dataset to 47.50.