Adversarial Dual-Student with Differentiable Spatial Warping for Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces a novel semi-supervised semantic segmentation method combining differentiable spatial warping for data augmentation and an adversarial dual-student framework to enhance model performance with limited labeled data.

Contribution

It proposes a context-friendly differentiable geometric warping for data augmentation and an adversarial dual-student framework to improve semi-supervised segmentation beyond existing methods.

Findings

Achieves state-of-the-art results on PASCAL VOC2012 and Cityscapes datasets.

Significantly improves performance with only 12.5% labeled data on PASCAL VOC2012.

Comparable mIoU of 73.4% to fully supervised models.

Abstract

A common challenge posed to robust semantic segmentation is the expensive data annotation cost. Existing semi-supervised solutions show great potential for solving this problem. Their key idea is constructing consistency regularization with unsupervised data augmentation from unlabeled data for model training. The perturbations for unlabeled data enable the consistency training loss, which benefits semi-supervised semantic segmentation. However, these perturbations destroy image context and introduce unnatural boundaries, which is harmful for semantic segmentation. Besides, the widely adopted semi-supervised learning framework, i.e. mean-teacher, suffers performance limitation since the student model finally converges to the teacher model. In this paper, first of all, we propose a context friendly differentiable geometric warping to conduct unsupervised data augmentation; secondly, a novel adversarial dual-student framework is proposed to improve the Mean-Teacher from the following two aspects: (1) dual student models are learned independently except for a stabilization constraint to encourage exploiting model diversities; (2) adversarial training scheme is applied to both students and the discriminators are resorted to distinguish reliable pseudo-label of unlabeled data for self-training. Effectiveness is validated via extensive experiments on PASCAL VOC2012 and Cityscapes. Our solution significantly improves the performance and state-of-the-art results are achieved on both datasets. Remarkably, compared with fully supervision, our solution achieves comparable mIoU of 73.4% using only 12.5% annotated data on PASCAL VOC2012. Our codes and models are available at https://github.com/cao-cong/ADS-SemiSeg.