LLM Safety From Within: Detecting Harmful Content with Internal Representations

TL;DR

SIREN is a lightweight guard model that leverages internal representations of LLMs to detect harmful content more effectively and efficiently than existing methods, without altering the original models.

Contribution

The paper introduces SIREN, a novel approach that uses internal LLM features and safety neurons to improve harmful content detection without model modification.

Findings

SIREN outperforms existing open-source guard models on multiple benchmarks.

SIREN uses 250 times fewer trainable parameters than state-of-the-art models.

SIREN generalizes well to unseen benchmarks and enables real-time detection.

Abstract

Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.