Debiasing, calibrating, and improving Semi-supervised Learning performance via simple Ensemble Projector

TL;DR

This paper introduces EPASS, a simple ensemble projector method that enhances semi-supervised learning by improving embeddings, leading to significant performance gains across various datasets and models.

Contribution

EPASS is a novel ensemble-based approach that improves semi-supervised learning performance by enhancing learned embeddings with multiple projectors, simplifying the design while boosting generalization.

Findings

EPASS improves baseline semi-supervised methods by up to 39.47% in top-1 error reduction.

EPASS achieves consistent performance gains across different datasets and architectures.

The method enhances feature representations, leading to better generalization in SSL tasks.

Abstract

Recent studies on semi-supervised learning (SSL) have achieved great success. Despite their promising performance, current state-of-the-art methods tend toward increasingly complex designs at the cost of introducing more network components and additional training procedures. In this paper, we propose a simple method named Ensemble Projectors Aided for Semi-supervised Learning (EPASS), which focuses mainly on improving the learned embeddings to boost the performance of the existing contrastive joint-training semi-supervised learning frameworks. Unlike standard methods, where the learned embeddings from one projector are stored in memory banks to be used with contrastive learning, EPASS stores the ensemble embeddings from multiple projectors in memory banks. As a result, EPASS improves generalization, strengthens feature representation, and boosts performance. For instance, EPASS improves strong baselines for semi-supervised learning by 39.47\%/31.39\%/24.70\% top-1 error rate, while using only 100k/1\%/10\% of labeled data for SimMatch, and achieves 40.24\%/32.64\%/25.90\% top-1 error rate for CoMatch on the ImageNet dataset. These improvements are consistent across methods, network architectures, and datasets, proving the general effectiveness of the proposed methods. Code is available at https://github.com/beandkay/EPASS.