Exploiting LLM Quantization

TL;DR

This paper uncovers a security vulnerability in LLM quantization, showing how malicious models can be hidden in benign full-precision models and only become harmful after quantization, posing a significant threat to deployment safety.

Contribution

It introduces the first study of security risks in LLM quantization, demonstrating a novel attack framework that exploits quantization constraints to embed malicious behavior.

Findings

Demonstrates feasibility of malicious quantized LLMs in three scenarios

Shows full-precision models can appear benign but become harmful after quantization

Highlights potential security risks in deploying quantized LLMs on public platforms

Abstract

Quantization leverages lower-precision weights to reduce the memory usage of large language models (LLMs) and is a key technique for enabling their deployment on commodity hardware. While LLM quantization's impact on utility has been extensively explored, this work for the first time studies its adverse effects from a security perspective. We reveal that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model. We demonstrate this threat using a three-staged attack framework: (i) first, we obtain a malicious LLM through fine-tuning on an adversarial task; (ii) next, we quantize the malicious model and calculate constraints that characterize all full-precision models that map to the same quantized model; (iii) finally, using projected gradient descent, we tune out the poisoned behavior from the full-precision model while ensuring that its weights satisfy the constraints computed in step (ii). This procedure results in an LLM that exhibits benign behavior in full precision but when quantized, it follows the adversarial behavior injected in step (i). We experimentally demonstrate the feasibility and severity of such an attack across three diverse scenarios: vulnerable code generation, content injection, and over-refusal attack. In practice, the adversary could host the resulting full-precision model on an LLM community hub such as Hugging Face, exposing millions of users to the threat of deploying its malicious quantized version on their devices.