On the Relationship between Skill Neurons and Robustness in Prompt Tuning

TL;DR

This paper investigates the relationship between skill neurons and the robustness of prompt tuning in large language models, finding that models with more consistent skill neuron activation tend to be more adversarially robust.

Contribution

It demonstrates the existence of skill neurons in T5, compares their robustness to RoBERTa, and links consistent skill neuron activation to adversarial robustness.

Findings

Prompt tuning prompts are transferable within task types.

Prompts for RoBERTa are less robust to adversarial data.

Prompts for T5 show slightly better robustness and skill neuron consistency.

Abstract

Prompt Tuning is a popular parameter-efficient finetuning method for pre-trained large language models (PLMs). Based on experiments with RoBERTa, it has been suggested that Prompt Tuning activates specific neurons in the transformer's feed-forward networks, that are highly predictive and selective for the given task. In this paper, we study the robustness of Prompt Tuning in relation to these "skill neurons", using RoBERTa and T5. We show that prompts tuned for a specific task are transferable to tasks of the same type but are not very robust to adversarial data. While prompts tuned for RoBERTa yield below-chance performance on adversarial data, prompts tuned for T5 are slightly more robust and retain above-chance performance in two out of three cases. At the same time, we replicate the finding that skill neurons exist in RoBERTa and further show that skill neurons also exist in T5. Interestingly, the skill neurons of T5 determined on non-adversarial data are also among the most predictive neurons on the adversarial data, which is not the case for RoBERTa. We conclude that higher adversarial robustness may be related to a model's ability to consistently activate the relevant skill neurons on adversarial data.