Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards

TL;DR

Clip-Tuning introduces a derivative-free prompt learning method using a mixture of rewards from thinned, frozen networks of large pre-trained models, improving performance in few-shot language understanding tasks.

Contribution

The paper proposes a novel Clip-Tuning approach that leverages diverse frozen sub-networks of PLMs to enhance derivative-free prompt learning.

Findings

Outperforms previous gradient-free methods

Achieves parity with gradient-based methods on benchmarks

Effective in few-shot learning scenarios

Abstract

Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did not take full advantage of the over-parameterized characteristics of large pre-trained language models (PLMs). In this paper, we propose Clip-Tuning, a simple yet effective method that adopts diverse frozen "thinned" networks of PLMs to obtain a mixture of rewards and thus advance the derivative-free prompt learning. The thinned networks consist of all the hidden units that survive a stationary dropout strategy, whose inference predictions reflect an ensemble of partial views over prompted training samples. Our method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings.