RTL-Breaker: Assessing the Security of LLMs against Backdoor Attacks on HDL Code Generation

TL;DR

This paper introduces RTL-Breaker, a framework to assess the security of LLMs in HDL code generation against backdoor attacks, highlighting vulnerabilities and the need for robust defenses.

Contribution

We propose RTL-Breaker, the first comprehensive backdoor attack framework targeting LLM-based HDL code generation, analyzing trigger mechanisms and attack side-effects.

Findings

Different trigger mechanisms vary in effectiveness

Backdoor attacks can impact code quality

Open-source framework for security assessment

Abstract

Large language models (LLMs) have demonstrated remarkable potential with code generation/completion tasks for hardware design. In fact, LLM-based hardware description language (HDL) code generation has enabled the industry to realize complex designs more quickly, reducing the time and effort required in the development cycle. However, the increased reliance on such automation introduces critical security risks. Notably, given that LLMs have to be trained on vast datasets of codes that are typically sourced from publicly available repositories (often without thorough validation), LLMs are susceptible to so-called data poisoning or backdoor attacks. Here, attackers inject malicious code for the training data, which can be carried over into the HDL code generated by LLMs. This threat vector can compromise the security and integrity of entire hardware systems. In this work, we propose RTL-Breaker, a novel backdoor attack framework on LLM-based HDL code generation. RTL-Breaker provides an in-depth analysis for essential aspects of this novel problem: 1) various trigger mechanisms versus their effectiveness for inserting malicious modifications, and 2) side-effects by backdoor attacks on code generation in general, i.e., impact on code quality. RTL-Breaker emphasizes the urgent need for more robust measures to safeguard against such attacks. Toward that end, we open-source our framework and all data.