IPixMatch: Boost Semi-supervised Semantic Segmentation with Inter-Pixel Relation

TL;DR

IPixMatch enhances semi-supervised semantic segmentation by effectively utilizing inter-pixel relations, leading to improved performance especially in low-data scenarios without requiring significant modifications to existing frameworks.

Contribution

It introduces a novel inter-pixel relation loss into teacher-student networks, improving semi-supervised segmentation performance with minimal framework changes.

Findings

Consistent performance improvements across multiple benchmarks

Effective in low-data regimes by leveraging inter-pixel relations

Seamless integration into existing teacher-student frameworks

Abstract

The scarcity of labeled data in real-world scenarios is a critical bottleneck of deep learning's effectiveness. Semi-supervised semantic segmentation has been a typical solution to achieve a desirable tradeoff between annotation cost and segmentation performance. However, previous approaches, whether based on consistency regularization or self-training, tend to neglect the contextual knowledge embedded within inter-pixel relations. This negligence leads to suboptimal performance and limited generalization. In this paper, we propose a novel approach IPixMatch designed to mine the neglected but valuable Inter-Pixel information for semi-supervised learning. Specifically, IPixMatch is constructed as an extension of the standard teacher-student network, incorporating additional loss terms to capture inter-pixel relations. It shines in low-data regimes by efficiently leveraging the limited labeled data and extracting maximum utility from the available unlabeled data. Furthermore, IPixMatch can be integrated seamlessly into most teacher-student frameworks without the need of model modification or adding additional components. Our straightforward IPixMatch method demonstrates consistent performance improvements across various benchmark datasets under different partitioning protocols.