Feature Space Renormalization for Semi-supervised Learning

TL;DR

This paper introduces a feature space renormalization (FSR) technique for semi-supervised learning that improves discriminative feature learning by enforcing consistency in feature representations, enhancing existing SSL frameworks.

Contribution

It proposes a novel FSR mechanism and a dual-branch module that can be integrated into existing SSL methods to boost their performance.

Findings

FSR improves SSL accuracy on benchmark datasets.

The module adds no extra computational overhead.

Compatible with frameworks like CRMatch and FreeMatch.

Abstract

Semi-supervised learning (SSL) has been proven to be a powerful method for leveraging unlabeled data to alleviate models'dependence on large labeled datasets. The common framework among recent approaches is to train the model on a large amount of unlabeled data with consistency regularization to constrain the model predictions to be invariant to input perturbation. This paper proposes a feature space renormalizati-on (FSR) mechanism for SSL, which imposes consistency on feature representations rather than on labels to enable the model to learn better discriminative features. In order to apply this mechanism to SSL, we design a dual-branch FSR module consisting of a dual-branch header and an FSR block. This module can be seamlessly plugged and played into existing SSL frameworks to enhance the performance of the base SSL. The experimental results show that our proposed FSR module helps the base SSL framework (e.g. CRMatch and FreeMatch), achieve better performance on a variety of standard SSL benchmark datasets, without incurring additional overhead in terms of computation time and GPU memory.