GenTel-Safe: A Unified Benchmark and Shielding Framework for Defending Against Prompt Injection Attacks

TL;DR

GenTel-Safe introduces a comprehensive framework with a novel attack detection method and an extensive benchmark to evaluate and improve defenses against prompt injection attacks in large language models.

Contribution

It presents GenTel-Shield for attack detection and GenTel-Bench for evaluation, addressing vulnerabilities and gaps in existing safety mechanisms for LLMs.

Findings

GenTel-Shield achieves state-of-the-art detection rates.

Existing safety guardrails are vulnerable to prompt injection.

The benchmark includes over 84,800 attack scenarios.

Abstract

Large Language Models (LLMs) like GPT-4, LLaMA, and Qwen have demonstrated remarkable success across a wide range of applications. However, these models remain inherently vulnerable to prompt injection attacks, which can bypass existing safety mechanisms, highlighting the urgent need for more robust attack detection methods and comprehensive evaluation benchmarks. To address these challenges, we introduce GenTel-Safe, a unified framework that includes a novel prompt injection attack detection method, GenTel-Shield, along with a comprehensive evaluation benchmark, GenTel-Bench, which compromises 84812 prompt injection attacks, spanning 3 major categories and 28 security scenarios. To prove the effectiveness of GenTel-Shield, we evaluate it together with vanilla safety guardrails against the GenTel-Bench dataset. Empirically, GenTel-Shield can achieve state-of-the-art attack detection success rates, which reveals the critical weakness of existing safeguarding techniques against harmful prompts. For reproducibility, we have made the code and benchmarking dataset available on the project page at https://gentellab.github.io/gentel-safe.github.io/.