Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach

TL;DR

This paper introduces COSINE, a contrastive self-training framework that enables fine-tuning pre-trained language models using only weak supervision, effectively reducing overfitting and improving performance across multiple NLP tasks.

Contribution

The paper proposes a novel contrastive self-training approach with regularization and reweighting to fine-tune language models without labeled data, outperforming baselines on various benchmarks.

Findings

Outperforms strong baselines on 7 benchmarks across 6 tasks

Achieves performance comparable to fully-supervised fine-tuning

Effectively suppresses error propagation during training

Abstract

Fine-tuned pre-trained language models (LMs) have achieved enormous success in many natural language processing (NLP) tasks, but they still require excessive labeled data in the fine-tuning stage. We study the problem of fine-tuning pre-trained LMs using only weak supervision, without any labeled data. This problem is challenging because the high capacity of LMs makes them prone to overfitting the noisy labels generated by weak supervision. To address this problem, we develop a contrastive self-training framework, COSINE, to enable fine-tuning LMs with weak supervision. Underpinned by contrastive regularization and confidence-based reweighting, this contrastive self-training framework can gradually improve model fitting while effectively suppressing error propagation. Experiments on sequence, token, and sentence pair classification tasks show that our model outperforms the strongest baseline by large margins on 7 benchmarks in 6 tasks, and achieves competitive performance with fully-supervised fine-tuning methods.