Provable Defense Framework for LLM Jailbreaks via Noise-Augumented Alignment

TL;DR

This paper introduces a provable defense framework for Large Language Models against jailbreak attacks, combining certifiable robustness with a novel noise-augmented alignment technique to significantly reduce attack success rates while maintaining high utility.

Contribution

It proposes Certified Semantic Smoothing and Noise-Augmented Alignment Tuning, providing the first certifiable robustness framework for LLMs against adaptive jailbreaks.

Findings

Attack success rate reduced from 84.2% to 1.2%.

Benign utility maintained at 94.1%.

Outperforms character-level baselines significantly.

Abstract

Large Language Models (LLMs) remain vulnerable to adaptive jailbreaks that easily bypass empirical defenses like GCG. We propose a framework for certifiable robustness that shifts safety guarantees from single-pass inference to the statistical stability of an ensemble. We introduce Certified Semantic Smoothing (CSS) via Stratified Randomized Ablation, a technique that partitions inputs into immutable structural prompts and mutable payloads to derive rigorous lo norm guarantees using the Hypergeometric distribution. To resolve performance degradation on sparse contexts, we employ Noise-Augmented Alignment Tuning (NAAT), which transforms the base model into a semantic denoiser. Extensive experiments on Llama-3 show that our method reduces the Attack Success Rate of gradient-based attacks from 84.2% to 1.2% while maintaining 94.1% benign utility, significantly outperforming character-level baselines which degrade utility to 74.3%. This framework provides a deterministic certificate of safety, ensuring that a model remains robust against all adversarial variants within a provable radius.