Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner

TL;DR

This paper introduces Prompt-based Continued Pre-training (PCP), a method that enhances prompt-based fine-tuning of language models by combining instruction tuning with pre-training, leading to significant performance improvements across various NLP tasks.

Contribution

The paper proposes PCP, a novel pre-training approach that improves prompt-based fine-tuning performance, outperforming existing methods without requiring iterative processes or additional data augmentation.

Findings

PCP improves prompt-based fine-tuning performance by up to 20.1%.

Conventional continued pre-training does not always benefit downstream tasks.

PCP outperforms state-of-the-art semi-supervised methods with less complexity.

Abstract

Language models (LMs) trained on vast quantities of unlabelled data have greatly advanced the field of natural language processing (NLP). In this study, we re-visit the widely accepted notion in NLP that continued pre-training LMs on task-related texts improves the performance of fine-tuning (FT) in downstream tasks. Through experiments on eight single-sentence tasks and eight sentence-pair tasks in both semi-supervised and fully-supervised settings, we find that conventional continued pre-training does not consistently provide benefits and can even be detrimental for sentence-pair tasks or when prompt-based FT is used. To tackle these issues, we propose Prompt-based Continued Pre-training (PCP), which combines the idea of instruction tuning with conventional continued pre-training. Our approach aims to improve the performance of prompt-based FT by presenting both task-related texts and prompt templates to LMs through unsupervised pre-training objectives before fine-tuning for the target task. Our empirical evaluations on 21 benchmarks demonstrate that the PCP consistently improves the performance of state-of-the-art prompt-based FT approaches (up to 20.1% absolute) in both semi-supervised and fully-supervised settings, even with only hundreds of unlabelled examples. Additionally, prompt-based FT with the PCP outperforms state-of-the-art semi-supervised approaches with greater simplicity, eliminating the need for an iterative process and extra data augmentation. Our further analysis explores the performance lower bound of the PCP and reveals that the advantages of PCP persist across different sizes of models and datasets.