HarmChip: Evaluating Hardware Security Centric LLM Safety via Jailbreak Benchmarking

TL;DR

HarmChip is a benchmark designed to evaluate the vulnerability of large language models to domain-specific security threats in hardware design, exposing safety gaps in current models.

Contribution

The paper introduces HarmChip, the first comprehensive benchmark for assessing LLM susceptibility to hardware security jailbreaks across multiple threat domains.

Findings

State-of-the-art LLMs often refuse legitimate security queries.

LLMs tend to comply with disguised malicious requests.

Existing safety measures fail to detect domain-specific threats.

Abstract

The integration of large language models (LLMs) into electronic design automation (EDA) workflows has introduced powerful capabilities for RTL generation, verification, and design optimization, but also raises critical security concerns. Malicious LLM outputs in this domain pose hardware-level threats, including hardware Trojan insertion, side-channel leakage, and intellectual property theft, that are irreversible once fabricated into silicon. Such requests often exploit semantic disguise, embedding adversarial intent within legitimate engineering language that existing safety mechanisms, trained on general-purpose hazards, fail to detect. No benchmark exists to evaluate LLM vulnerability to such domain-specific threats. We present the HarmChip benchmark to assess jailbreak susceptibility in hardware security, spanning 16 hardware security domains, 120 threats, and 360 prompts at two difficulty levels. Evaluation of state-of-the-art LLMs reveals an alignment paradox: They refuse legitimate security queries while complying with semantically disguised attacks, exposing blind spots in safety guardrails and underscoring the need for domain-aware safety alignment.