AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks

TL;DR

AlignSentinel is a novel three-class classifier that detects prompt injection attacks by considering instruction alignment, significantly improving detection accuracy over existing methods that misclassify benign aligned instructions as malicious.

Contribution

This work introduces AlignSentinel, the first classifier to distinguish among aligned, misaligned, and non-instruction inputs, utilizing attention-based features and a new benchmark dataset.

Findings

AlignSentinel accurately detects misaligned instructions.

It outperforms existing detection baselines.

The benchmark includes diverse input categories.

Abstract

Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one. Existing detection defenses typically classify any input with instruction as malicious, leading to misclassification of benign inputs containing instructions that align with the intended task. In this work, we account for the instruction hierarchy and distinguish among three categories: inputs with misaligned instructions, inputs with aligned instructions, and non-instruction inputs. We introduce AlignSentinel, a three-class classifier that leverages features derived from LLM's attention maps to categorize inputs accordingly. To support evaluation, we construct the first systematic benchmark containing inputs from all three categories. Experiments on both our benchmark and existing ones--where inputs with aligned instructions are largely absent--show that AlignSentinel accurately detects inputs with misaligned instructions and substantially outperforms baselines.