BadTemplate: A Training-Free Backdoor Attack via Chat Template Against Large Language Models

TL;DR

This paper introduces BadTemplate, a training-free backdoor attack method that manipulates chat templates in large language models to inject malicious instructions, achieving high success rates and exposing security vulnerabilities.

Contribution

The paper presents a novel, training-free backdoor attack leveraging chat template customizability to embed malicious prompts in LLMs, outperforming traditional methods in effectiveness and ease of deployment.

Findings

Achieves up to 100% attack success rate across multiple datasets and models.

Outperforms traditional prompt-based backdoors in effectiveness.

Detection methods are largely ineffective against BadTemplate.

Abstract

Chat template is a common technique used in the training and inference stages of Large Language Models (LLMs). It can transform input and output data into role-based and templated expressions to enhance the performance of LLMs. However, this also creates a breeding ground for novel attack surfaces. In this paper, we first reveal that the customizability of chat templates allows an attacker who controls the template to inject arbitrary strings into the system prompt without the user's notice. Building on this, we propose a training-free backdoor attack, termed BadTemplate. Specifically, BadTemplate inserts carefully crafted malicious instructions into the high-priority system prompt, thereby causing the target LLM to exhibit persistent backdoor behaviors. BadTemplate outperforms traditional backdoor attacks by embedding malicious instructions directly into the system prompt, eliminating the need for model retraining while achieving high attack effectiveness with minimal cost. Furthermore, its simplicity and scalability make it easily and widely deployed in real-world systems, raising serious risks of rapid propagation, economic damage, and large-scale misinformation. Furthermore, detection by major third-party platforms HuggingFace and LLM-as-a-judge proves largely ineffective against BadTemplate. Extensive experiments conducted on 5 benchmark datasets across 6 open-source and 3 closed-source LLMs, compared with 3 baselines, demonstrate that BadTemplate achieves up to a 100% attack success rate and significantly outperforms traditional prompt-based backdoors in both word-level and sentence-level attacks. Our work highlights the potential security risks raised by chat templates in the LLM supply chain, thereby supporting the development of effective defense mechanisms.