Linking data separation, visual separation, and classifier performance using pseudo-labeling by contrastive learning

TL;DR

This paper explores how contrastive learning improves pseudo-labeling in deep neural networks by enhancing data and visual separation, leading to better classifier performance on challenging biological image datasets.

Contribution

It introduces contrastive learning methods to produce better latent spaces for pseudo-labeling, establishing correlations between data separation, visual separation, and classifier accuracy.

Findings

Contrastive learning improves latent space quality for pseudo-labeling.

Visual separation correlates with classifier performance.

Effective classification achieved with only 1% labeled data.

Abstract

Lacking supervised data is an issue while training deep neural networks (DNNs), mainly when considering medical and biological data where supervision is expensive. Recently, Embedded Pseudo-Labeling (EPL) addressed this problem by using a non-linear projection (t-SNE) from a feature space of the DNN to a 2D space, followed by semi-supervised label propagation using a connectivity-based method (OPFSemi). We argue that the performance of the final classifier depends on the data separation present in the latent space and visual separation present in the projection. We address this by first proposing to use contrastive learning to produce the latent space for EPL by two methods (SimCLR and SupCon) and by their combination, and secondly by showing, via an extensive set of experiments, the aforementioned correlations between data separation, visual separation, and classifier performance. We demonstrate our results by the classification of five real-world challenging image datasets of human intestinal parasites with only 1% supervised samples.