GuidedMix-Net: Semi-supervised Semantic Segmentation by Using Labeled Images as Reference

TL;DR

GuidedMix-Net introduces a semi-supervised semantic segmentation approach that leverages labeled data to guide unlabeled data learning through image pair interpolation, mutual information transfer, and pseudo mask generalization, improving accuracy.

Contribution

The paper presents a novel GuidedMix-Net method that effectively utilizes labeled information to enhance unlabeled data segmentation, outperforming previous semi-supervised approaches.

Findings

Achieves +7% mIoU improvement on PASCAL VOC 2012 and Cityscapes.

Effectively learns higher-quality pseudo masks for unlabeled data.

Demonstrates competitive segmentation accuracy through extensive experiments.

Abstract

Semi-supervised learning is a challenging problem which aims to construct a model by learning from limited labeled examples. Numerous methods for this task focus on utilizing the predictions of unlabeled instances consistency alone to regularize networks. However, treating labeled and unlabeled data separately often leads to the discarding of mass prior knowledge learned from the labeled examples. %, and failure to mine the feature interaction between the labeled and unlabeled image pairs. In this paper, we propose a novel method for semi-supervised semantic segmentation named GuidedMix-Net, by leveraging labeled information to guide the learning of unlabeled instances. Specifically, GuidedMix-Net employs three operations: 1) interpolation of similar labeled-unlabeled image pairs; 2) transfer of mutual information; 3) generalization of pseudo masks. It enables segmentation models can learning the higher-quality pseudo masks of unlabeled data by transfer the knowledge from labeled samples to unlabeled data. Along with supervised learning for labeled data, the prediction of unlabeled data is jointly learned with the generated pseudo masks from the mixed data. Extensive experiments on PASCAL VOC 2012, and Cityscapes demonstrate the effectiveness of our GuidedMix-Net, which achieves competitive segmentation accuracy and significantly improves the mIoU by +7$\%$ compared to previous approaches.