JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification

TL;DR

JointMatch is a novel semi-supervised text classification method that reduces bias and error accumulation by adaptively thresholding and cross-teaching between two networks, achieving state-of-the-art results especially with scarce labels.

Contribution

The paper introduces JointMatch, a unified approach that combines adaptive classwise thresholding and dual-network teaching to improve semi-supervised text classification.

Findings

Achieves 5.13% higher accuracy on benchmark datasets.

Performs well with only 5 labels per class, reaching 86% accuracy on AG News.

Effectively mitigates pseudo-label bias and error accumulation.

Abstract

Semi-supervised text classification (SSTC) has gained increasing attention due to its ability to leverage unlabeled data. However, existing approaches based on pseudo-labeling suffer from the issues of pseudo-label bias and error accumulation. In this paper, we propose JointMatch, a holistic approach for SSTC that addresses these challenges by unifying ideas from recent semi-supervised learning and the task of learning with noise. JointMatch adaptively adjusts classwise thresholds based on the learning status of different classes to mitigate model bias towards current easy classes. Additionally, JointMatch alleviates error accumulation by utilizing two differently initialized networks to teach each other in a cross-labeling manner. To maintain divergence between the two networks for mutual learning, we introduce a strategy that weighs more disagreement data while also allowing the utilization of high-quality agreement data for training. Experimental results on benchmark datasets demonstrate the superior performance of JointMatch, achieving a significant 5.13% improvement on average. Notably, JointMatch delivers impressive results even in the extremely-scarce-label setting, obtaining 86% accuracy on AG News with only 5 labels per class. We make our code available at https://github.com/HenryPengZou/JointMatch.