When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations

TL;DR

This paper explores how LLM backdoor attacks can be understood and detected by analyzing the explanations generated for model decisions, revealing distinct patterns for poisoned versus clean inputs.

Contribution

It introduces a novel approach using explanation generation to identify backdoor attacks in LLMs, providing new insights into their mechanisms and detection methods.

Findings

Backdoored models produce flawed explanations for poisoned data.

Explanation tokens for poisoned samples appear late in transformer layers.

Attention shifts away from input context during explanation for poisoned inputs.

Abstract

Large Language Models (LLMs) are known to be vulnerable to backdoor attacks, where triggers embedded in poisoned samples can maliciously alter LLMs' behaviors. In this paper, we move beyond attacking LLMs and instead examine backdoor attacks through the novel lens of natural language explanations. Specifically, we leverage LLMs' generative capabilities to produce human-readable explanations for their decisions, enabling direct comparisons between explanations for clean and poisoned samples. Our results show that backdoored models produce coherent explanations for clean inputs but diverse and logically flawed explanations for poisoned data, a pattern consistent across classification and generation tasks for different backdoor attacks. Further analysis reveals key insights into the explanation generation process. At the token level, explanation tokens associated with poisoned samples only appear in the final few transformer layers. At the sentence level, attention dynamics indicate that poisoned inputs shift attention away from the original input context during explanation generation. These findings enhance our understanding of backdoor mechanisms in LLMs and present a promising framework for detecting vulnerabilities through explainability.