Palisade -- Prompt Injection Detection Framework

TL;DR

Palisade introduces a layered NLP-based framework combining rule-based filtering, ML classification, and LLM analysis to detect prompt injection attacks, improving security in AI systems.

Contribution

The paper presents a novel multi-layer detection framework that enhances prompt injection detection accuracy over traditional static methods.

Findings

ML classifier achieves highest individual accuracy

Multi-layer approach reduces false negatives significantly

Framework prioritizes security despite increased false positives

Abstract

The advent of Large Language Models LLMs marks a milestone in Artificial Intelligence, altering how machines comprehend and generate human language. However, LLMs are vulnerable to malicious prompt injection attacks, where crafted inputs manipulate the models behavior in unintended ways, compromising system integrity and causing incorrect outcomes. Conventional detection methods rely on static, rule-based approaches, which often fail against sophisticated threats like abnormal token sequences and alias substitutions, leading to limited adaptability and higher rates of false positives and false negatives.This paper proposes a novel NLP based approach for prompt injection detection, emphasizing accuracy and optimization through a layered input screening process. In this framework, prompts are filtered through three distinct layers rule-based, ML classifier, and companion LLM before reaching the target model, thereby minimizing the risk of malicious interaction.Tests show the ML classifier achieves the highest accuracy among individual layers, yet the multi-layer framework enhances overall detection accuracy by reducing false negatives. Although this increases false positives, it minimizes the risk of overlooking genuine injected prompts, thus prioritizing security.This multi-layered detection approach highlights LLM vulnerabilities and provides a comprehensive framework for future research, promoting secure interactions between humans and AI systems.