Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment

TL;DR

Risky-Bench is a comprehensive framework for evaluating safety risks of large language model agents in real-world deployments, addressing limitations of existing methods by providing systematic, adaptable, and context-aware safety assessments.

Contribution

It introduces a domain-agnostic, systematic evaluation framework that assesses agent safety risks across diverse real-world scenarios and threat models.

Findings

Uncovered safety risks in state-of-the-art agents during realistic tasks.

Demonstrated adaptability of Risky-Bench to various deployment settings.

Provided a structured methodology for comprehensive safety evaluation.

Abstract

Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm. Existing agent safety evaluations rely on risk-oriented tasks tailored to specific agent settings, resulting in limited coverage of safety risk space and failing to assess agent safety behavior during long-horizon, interactive task execution in complex real-world deployments. Moreover, their specialization to particular agent settings limits adaptability across diverse agent configurations. To address these limitations, we propose Risky-Bench, a framework that enables systematic agent safety evaluation grounded in real-world deployment. Risky-Bench organizes evaluation around domain-agnostic safety principles to derive context-aware safety rubrics that delineate safety space, and systematically evaluates safety risks across this space through realistic task execution under varying threat assumptions. When applied to life-assist agent settings, Risky-Bench uncovers substantial safety risks in state-of-the-art agents under realistic execution conditions. Moreover, as a well-structured evaluation pipeline, Risky-Bench is not confined to life-assist scenarios and can be adapted to other deployment settings to construct environment-specific safety evaluations, providing an extensible methodology for agent safety assessment.