Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation

TL;DR

BAGEL is a modular ensemble framework that efficiently detects malicious prompts in LLMs, adapting incrementally to new threats with high accuracy and interpretability, outperforming large-scale models in safety tasks.

Contribution

The paper introduces BAGEL, a lightweight, incrementally updatable ensemble system that improves malicious prompt detection in LLMs, balancing performance, efficiency, and adaptability.

Findings

BAGEL achieves an F1 score of 0.92 with only 5 ensemble members.

It outperforms OpenAI Moderation API and ShieldGemma in detection accuracy.

BAGEL maintains robustness after nine incremental updates.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding, reasoning, and generation. However, these systems remain susceptible to malicious prompts that induce unsafe or policy-violating behavior through harmful requests, jailbreak techniques, and prompt injection attacks. Existing defenses face fundamental limitations: black-box moderation APIs offer limited transparency and adapt poorly to evolving threats, while white-box approaches using large LLM judges impose prohibitive computational costs and require expensive retraining for new attacks. Current systems force designers to choose between performance, efficiency, and adaptability. To address these challenges, we present BAGEL (Bootstrap AGgregated Ensemble Layer), a modular, lightweight, and incrementally updatable framework for malicious prompt detection. BAGEL employs a bootstrap aggregation and mixture of expert inspired ensemble of fine-tuned models, each specialized on a different attack dataset. At inference, BAGEL uses a random forest router to identify the most suitable ensemble member, then applies stochastic selection to sample additional members for prediction aggregation. When new attacks emerge, BAGEL updates incrementally by fine-tuning a small prompt-safety classifier (86M parameters) and adding the resulting model to the ensemble. BAGEL achieves an F1 score of 0.92 by selecting just 5 ensemble members (430M parameters), outperforming OpenAI Moderation API and ShieldGemma which require billions of parameters. Performance remains robust after nine incremental updates, and BAGEL provides interpretability through its router's structural features. Our results show ensembles of small finetuned classifiers can match or exceed billion-parameter guardrails while offering the adaptability and efficiency required for production systems.