Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks

TL;DR

This study assesses the robustness of large language model safety guardrails against adversarial attacks, revealing significant performance gaps on unseen prompts and identifying new failure modes, emphasizing the need for better evaluation metrics.

Contribution

It provides a comprehensive evaluation of ten guardrail models across diverse attack categories, highlighting their limitations and proposing generalization ability as a key metric.

Findings

All models perform poorly on unseen prompts.

Qwen3Guard's accuracy drops from 91.0% to 33.8%.

Some models generate harmful content in helpful mode.

Abstract

Large Language Model (LLM) safety guardrail models have emerged as a primary defense mechanism against harmful content generation, yet their robustness against sophisticated adversarial attacks remains poorly characterized. This study evaluated ten publicly available guardrail models from Meta, Google, IBM, NVIDIA, Alibaba, and Allen AI across 1,445 test prompts spanning 21 attack categories. While Qwen3Guard-8B achieved the highest overall accuracy (85.3%, 95% CI: 83.4-87.1%), a critical finding emerged when separating public benchmark prompts from novel attacks: all models showed substantial performance degradation on unseen prompts, with Qwen3Guard dropping from 91.0% to 33.8% (a 57.2 percentage point gap). In contrast, Granite-Guardian-3.2-5B showed the best generalization with only a 6.5% gap. A "helpful mode" jailbreak was also discovered where two guardrail models (Nemotron-Safety-8B, Granite-Guardian-3.2-5B) generated harmful content instead of blocking it, representing a novel failure mode. These findings suggest that benchmark performance may be misleading due to training data contamination, and that generalization ability, not overall accuracy, should be the primary metric for guardrail evaluation.