Parameter-Efficient Transfer Learning with Diff Pruning

TL;DR

Diff pruning enables parameter-efficient transfer learning by learning sparse task-specific modifications to pretrained models, matching full finetuning performance with minimal parameter updates, suitable for multi-task and streaming scenarios.

Contribution

Introduces diff pruning, a method for sparse, parameter-efficient transfer learning that maintains performance while significantly reducing task-specific parameter storage.

Findings

Achieves GLUE benchmark performance comparable to full finetuning.

Modifies only 0.5% of parameters per task on average.

Requires storing only sparse diff vectors for each task.

Abstract

While task-specific finetuning of pretrained networks has led to significant empirical advances in NLP, the large size of networks makes finetuning difficult to deploy in multi-task, memory-constrained settings. We propose diff pruning as a simple approach to enable parameter-efficient transfer learning within the pretrain-finetune framework. This approach views finetuning as learning a task-specific diff vector that is applied on top of the pretrained parameter vector, which remains fixed and is shared across different tasks. The diff vector is adaptively pruned during training with a differentiable approximation to the L0-norm penalty to encourage sparsity. Diff pruning becomes parameter-efficient as the number of tasks increases, as it requires storing only the nonzero positions and weights of the diff vector for each task, while the cost of storing the shared pretrained model remains constant. It further does not require access to all tasks during training, which makes it attractive in settings where tasks arrive in stream or the set of tasks is unknown. We find that models finetuned with diff pruning can match the performance of fully finetuned baselines on the GLUE benchmark while only modifying 0.5% of the pretrained model's parameters per task.