Embedding-based classifiers can detect prompt injection attacks

TL;DR

This paper introduces embedding-based machine learning classifiers, particularly Random Forest and XGBoost, to effectively detect prompt injection attacks on large language models, outperforming existing methods.

Contribution

The paper presents a novel embedding-based classification approach that improves detection of prompt injection attacks over prior neural network methods.

Findings

Embedding-based classifiers outperform existing prompt injection detection methods.

Random Forest and XGBoost classifiers achieve the best performance.

The approach effectively distinguishes malicious prompts from benign ones.

Abstract

Large Language Models (LLMs) are seeing significant adoption in every type of organization due to their exceptional generative capabilities. However, LLMs are found to be vulnerable to various adversarial attacks, particularly prompt injection attacks, which trick them into producing harmful or inappropriate content. Adversaries execute such attacks by crafting malicious prompts to deceive the LLMs. In this paper, we propose a novel approach based on embedding-based Machine Learning (ML) classifiers to protect LLM-based applications against this severe threat. We leverage three commonly used embedding models to generate embeddings of malicious and benign prompts and utilize ML classifiers to predict whether an input prompt is malicious. Out of several traditional ML methods, we achieve the best performance with classifiers built using Random Forest and XGBoost. Our classifiers outperform state-of-the-art prompt injection classifiers available in open-source implementations, which use encoder-only neural networks.