How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation

TL;DR

This paper introduces DAPRO, a dynamic budget allocation framework that provides reliable bounds on the number of interactions needed to trigger key events in multi-turn LLM evaluations, improving efficiency and accuracy.

Contribution

DAPRO is the first theoretically valid dynamic allocation method for bounding time-to-event in multi-turn LLM interactions, offering tighter guarantees and unbiased estimates under limited resources.

Findings

DAPRO achieves coverage closer to the nominal level with lower variance.

DAPRO outperforms static baselines in various LLM evaluation tasks.

Theoretical bounds scale with the square root of mean censoring weight, not worst-case.

Abstract

Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events -- e.g., jailbreaks or successful task completion by an agent -- often emerge only after repeated interactions. These events might be rare, and under any feasible computational budget, remain unobserved. Recent conformal survival frameworks construct reliable lower predictive bounds (LPBs) on the number of iterations to trigger the event of interest, but rely on static budget allocation that is inefficient in multi-turn setups. To address this, we introduce \emph{Dynamic Allocation via PRojected Optimization} (DAPRO), the first theoretically valid dynamic budget allocation framework for bounding the time-to-event in multi-turn LLM interactions. We prove that DAPRO satisfies the budget constraint and provides distribution-free, finite-sample coverage guarantees without requiring the conditional independence between censoring and event times assumed by prior conformal survival approaches. A key theoretical contribution is a novel coverage bound that scales with the square root of the mean censoring weight rather than the worst-case weight, yielding provably tighter guarantees than prior work. Furthermore, DAPRO can be employed to obtain unbiased, low-variance estimates of population-level evaluation metrics, such as the jailbreak rate, under limited computing resources. Comprehensive experiments across agentic task success, adversarial jailbreaks, toxic content generation, and RAG hallucinations using LLMs such as Llama 3.1 and Qwen 2.5 demonstrate that DAPRO consistently achieves coverage closer to the nominal level with lower variance than static baselines, while satisfying the budget constraint.