Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts

TL;DR

This paper introduces Late Prompt Tuning (LPT), a novel method that inserts a prompt into an intermediate layer of pre-trained models, achieving competitive performance with faster training and lower memory usage compared to existing prompt tuning methods.

Contribution

The paper proposes a new LPT method that inserts a neural prompt into an intermediate layer, improving efficiency and performance over traditional prompt tuning methods.

Findings

LPT achieves performance comparable to full model tuning.

LPT has faster training speed and lower memory cost.

LPT performs well in both full-data and few-shot scenarios.

Abstract

Prompt tuning is a parameter-efficient tuning (PETuning) method for utilizing pre-trained models (PTMs) that simply prepends a soft prompt to the input and only optimizes the prompt to adapt PTMs to downstream tasks. Although it is parameter- and deployment-efficient, its performance still lags behind other state-of-the-art PETuning methods. Besides, the training cost of prompt tuning is not significantly reduced due to the back-propagation through the entire model. Through empirical analyses, we shed some light on the lagging performance of prompt tuning and recognize a trade-off between the propagation distance from label signals to the inserted prompt and the influence of the prompt on model outputs. Further, we present Late Prompt Tuning (LPT) that inserts a late prompt into an intermediate layer of the PTM instead of the input layer or all layers. The late prompt is obtained by a neural prompt generator conditioned on the hidden states before the prompt insertion layer and therefore is instance-dependent. Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.