"Energon": Unveiling Transformers from GPU Power and Thermal Side-Channels

TL;DR

This paper uncovers GPU power and thermal side-channels as a new attack vector to extract architectural details from pre-trained transformer models, revealing significant security vulnerabilities in shared GPU environments.

Contribution

It is the first study to analyze GPU power and thermal fluctuations as side-channels for extracting transformer model architecture information.

Findings

Achieves over 89% accuracy in model family identification

100% accuracy in hyperparameter classification

Over 93% success rate in transfer adversarial attacks

Abstract

Transformers have become the backbone of many Machine Learning (ML) applications, including language translation, summarization, and computer vision. As these models are increasingly deployed in shared Graphics Processing Unit (GPU) environments via Machine Learning as a Service (MLaaS), concerns around their security grow. In particular, the risk of side-channel attacks that reveal architectural details without physical access remains underexplored, despite the high value of the proprietary models they target. This work to the best of our knowledge is the first to investigate GPU power and thermal fluctuations as side-channels and further exploit them to extract information from pre-trained transformer models. The proposed analysis shows how these side channels can be exploited at user-privilege to reveal critical architectural details such as encoder/decoder layer and attention head for both language and vision transformers. We demonstrate the practical impact by evaluating multiple language and vision pre-trained transformers which are publicly available. Through extensive experimental evaluations, we demonstrate that the attack model achieves a high accuracy of over 89% on average for model family identification and 100% for hyperparameter classification, in both single-process as well as noisy multi-process scenarios. Moreover, by leveraging the extracted architectural information, we demonstrate highly effective black-box transfer adversarial attacks with an average success rate exceeding 93%, underscoring the security risks posed by GPU side-channel leakage in deployed transformer models.