HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

TL;DR

HiddenDetect is a novel, tuning-free framework that detects jailbreak attacks on large vision-language models by monitoring internal activation patterns, offering an efficient and scalable safety enhancement without extensive fine-tuning.

Contribution

We introduce HiddenDetect, a new method leveraging internal model activations to detect unsafe prompts in LVLMs without additional training.

Findings

Outperforms state-of-the-art jailbreak detection methods

Effectively identifies unsafe prompts via activation pattern analysis

Provides a scalable solution for LVLM safety enhancement

Abstract

The integration of additional modalities increases the susceptibility of large vision-language models (LVLMs) to safety risks, such as jailbreak attacks, compared to their language-only counterparts. While existing research primarily focuses on post-hoc alignment techniques, the underlying safety mechanisms within LVLMs remain largely unexplored. In this work , we investigate whether LVLMs inherently encode safety-relevant signals within their internal activations during inference. Our findings reveal that LVLMs exhibit distinct activation patterns when processing unsafe prompts, which can be leveraged to detect and mitigate adversarial inputs without requiring extensive fine-tuning. Building on this insight, we introduce HiddenDetect, a novel tuning-free framework that harnesses internal model activations to enhance safety. Experimental results show that {HiddenDetect} surpasses state-of-the-art methods in detecting jailbreak attacks against LVLMs. By utilizing intrinsic safety-aware patterns, our method provides an efficient and scalable solution for strengthening LVLM robustness against multimodal threats. Our code will be released publicly at https://github.com/leigest519/HiddenDetect.