Exploiting Minority Pseudo-Labels for Semi-Supervised Fine-grained Road Scene Understanding

TL;DR

This paper introduces a semi-supervised learning approach that leverages minority pseudo-labels and a novel mismatch score to improve recognition of rare classes in fine-grained road scene segmentation, enhancing balanced performance.

Contribution

It proposes a dual-module training framework that learns from all pseudo-labels and specifically from reliable minority class labels, with a contrastive learning strategy to balance class representation.

Findings

Outperforms traditional methods in recognizing tail classes.

Effectively balances class recognition in semi-supervised segmentation.

Demonstrates improved performance on multiple public benchmarks.

Abstract

In fine-grained road scene understanding, semantic segmentation plays a crucial role in enabling vehicles to perceive and comprehend their surroundings. By assigning a specific class label to each pixel in an image, it allows for precise identification and localization of detailed road features, which is vital for high-quality scene understanding and downstream perception tasks. A key challenge in this domain lies in improving the recognition performance of minority classes while mitigating the dominance of majority classes, which is essential for achieving balanced and robust overall performance. However, traditional semi-supervised learning methods often train models overlooking the imbalance between classes. To address this issue, firstly, we propose a general training module that learns from all the pseudo-labels without a conventional filtering strategy. Secondly, we propose a professional training module to learn specifically from reliable minority-class pseudo-labels identified by a novel mismatch score metric. The two modules are crossly supervised by each other so that it reduces model coupling which is essential for semi-supervised learning. During contrastive learning, to avoid the dominance of the majority classes in the feature space, we propose a strategy to assign evenly distributed anchors for different classes in the feature space. Experimental results on multiple public benchmarks show that our method surpasses traditional approaches in recognizing tail classes.