CourtGuard: A Local, Multiagent Prompt Injection Classifier

TL;DR

CourtGuard introduces a multiagent, court-like system for classifying prompt injections in LLMs, emphasizing lower false positives and advancing multiagent defense strategies despite some limitations in detection accuracy.

Contribution

This paper presents CourtGuard, a novel multiagent prompt injection classifier that uses a court-like system to improve false positive rates in prompt injection detection.

Findings

Lower false positive rate than the Direct Detector

Highlights importance of adversarial and benign scenario consideration

Advances multiagent system use in prompt injection defense

Abstract

As large language models (LLMs) become integrated into various sensitive applications, prompt injection, the use of prompting to induce harmful behaviors from LLMs, poses an ever increasing risk. Prompt injection attacks can cause LLMs to leak sensitive data, spread misinformation, and exhibit harmful behaviors. To defend against these attacks, we propose CourtGuard, a locally-runnable, multiagent prompt injection classifier. In it, prompts are evaluated in a court-like multiagent LLM system, where a "defense attorney" model argues the prompt is benign, a "prosecution attorney" model argues the prompt is a prompt injection, and a "judge" model gives the final classification. CourtGuard has a lower false positive rate than the Direct Detector, an LLM as-a-judge. However, CourtGuard is generally a worse prompt injection detector. Nevertheless, this lower false positive rate highlights the importance of considering both adversarial and benign scenarios for the classification of a prompt. Additionally, the relative performance of CourtGuard in comparison to other prompt injection classifiers advances the use of multiagent systems as a defense against prompt injection attacks. The implementations of CourtGuard and the Direct Detector with full prompts for Gemma-3-12b-it, Llama-3.3-8B, and Phi-4-mini-instruct are available at https://github.com/isaacwu2000/CourtGuard.