Quantamination: Dynamic Quantization Leaks Your Data Across the Batch

TL;DR

This paper uncovers a privacy vulnerability called Quantamination, where dynamic quantization in machine learning frameworks can leak sensitive data across batch boundaries, risking user privacy during model serving.

Contribution

The paper identifies and analyzes a critical side-channel vulnerability in dynamic quantization, demonstrating its potential to leak user data across batch boundaries in popular ML frameworks.

Findings

At least 4 popular ML frameworks leak data via Quantamination.

Dynamic quantization can be exploited to partially or fully recover other users' batched data.

The vulnerability arises from improper implementation or configuration of dynamic quantization.

Abstract

Dynamic quantization emerged as a practical approach to increase the utilization and efficiency of the machine learning serving flow. Unlike static quantization, which applies quantization offline, dynamic quantization operates on tensors at run-time, adapting its parameters to the actual input data. Today's mainstream machine learning frameworks, including ML compilers and inference engines, frequently recommend dynamic quantization as an initial step for optimizing model serving. This is because dynamic quantization can significantly reduce memory usage and computational load, leading to faster token generation and improved model serving efficiency without substantial loss in model accuracy. In this paper, we reveal a critical vulnerability in dynamic quantization: an adversary can exploit such quantization strategy to steal sensitive user data placed in the same batch as the adversary's input. Our analysis demonstrates that dynamic quantization, when improperly implemented or configured, can create side channels that expose information about other inputs within the same batch. We call this phenomenon Quantamination, describing contamination from quantization. Specifically, we show that at least 4 of the most popular ML frameworks in use today either default to or can use configurations that leak data across the batch boundary. This data leakage, in theory, allows attackers to partially or even fully recover other users' batched input data, representing a serious privacy risk for existing ML serving frameworks.