Measuring Impacts of Poisoning on Model Parameters and Neuron Activations: A Case Study of Poisoning CodeBERT

TL;DR

This paper investigates how poisoning attacks affect CodeBERT's internal parameters and activations, revealing patterns in some features that could aid in detecting backdoors in code language models.

Contribution

It provides a detailed analysis of parameter and activation changes in poisoned CodeBERT models, highlighting potential signals for backdoor detection.

Findings

Activation values and context embeddings show noticeable patterns in poisoned models

Attention weights and biases do not significantly differ between clean and poisoned models

Analysis aids in white-box detection of backdoors in code language models

Abstract

Large language models (LLMs) have revolutionized software development practices, yet concerns about their safety have arisen, particularly regarding hidden backdoors, aka trojans. Backdoor attacks involve the insertion of triggers into training data, allowing attackers to manipulate the behavior of the model maliciously. In this paper, we focus on analyzing the model parameters to detect potential backdoor signals in code models. Specifically, we examine attention weights and biases, activation values, and context embeddings of the clean and poisoned CodeBERT models. Our results suggest noticeable patterns in activation values and context embeddings of poisoned samples for the poisoned CodeBERT model; however, attention weights and biases do not show any significant differences. This work contributes to ongoing efforts in white-box detection of backdoor signals in LLMs of code through the analysis of parameters and activations.