Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing

TL;DR

This paper introduces a decoding-time safety-awareness probing method that detects unsafe content in large language models by leveraging latent safety signals, improving defense against jailbreak attacks without harming utility.

Contribution

The paper presents a novel decoding-time safety probing approach that activates intrinsic safety signals in LLMs to better detect and prevent jailbreak-generated unsafe content.

Findings

Significantly improves safety against jailbreak attacks

Maintains low false positive rates on benign inputs

Preserves response quality during detection

Abstract

Large language models (LLMs) have achieved impressive performance across natural language tasks and are increasingly deployed in real-world applications. Despite extensive safety alignment efforts, recent studies show that such alignment is often shallow and remains vulnerable to jailbreak attacks. Existing defense mechanisms, including decoding-based constraints and post-hoc content detectors, struggle against sophisticated jailbreaks, often intervening robust detection or excessively degrading model utility. In this work, we examine the decoding process of LLMs and make a key observation: even when successfully jailbroken, models internally exhibit latent safety-related signals during generation. However, these signals are overridden by the model's drive for fluent continuation, preventing timely self-correction or refusal. Building on this observation, we propose a simple yet effective approach that explicitly surfaces and leverages these latent safety signals for early detection of unsafe content during decoding. Experiments across diverse jailbreak attacks demonstrate that our approach significantly enhances safety, while maintaining low over-refusal rates on benign inputs and preserving response quality. Our results suggest that activating intrinsic safety-awareness during decoding offers a promising and complementary direction for defending against jailbreak attacks. Code is available at: https://github.com/zyz13590/SafeProbing.