Detecting Prompt Injection Attacks Against Application Using Classifiers

TL;DR

This paper develops and evaluates machine learning classifiers to detect prompt injection attacks in web applications, enhancing security measures for large language model integrations.

Contribution

It introduces a dataset for prompt injection detection and compares multiple classifiers to improve attack identification.

Findings

LSTM and neural networks outperform traditional models

The dataset augmentation enhances detection accuracy

Classifiers effectively identify malicious prompts

Abstract

Prompt injection attacks can compromise the security and stability of critical systems, from infrastructure to large web applications. This work curates and augments a prompt injection dataset based on the HackAPrompt Playground Submissions corpus and trains several classifiers, including LSTM, feed forward neural networks, Random Forest, and Naive Bayes, to detect malicious prompts in LLM integrated web applications. The proposed approach improves prompt injection detection and mitigation, helping protect targeted applications and systems.