AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions

TL;DR

SAFE is a comprehensive benchmark suite designed to evaluate the safety of embodied vision-language models in hazardous scenarios, revealing critical safety failures and guiding improvements in embodied agent safety.

Contribution

The paper introduces SAFE, a multi-component benchmark for systematic safety assessment of embodied VLM agents, covering perception, planning, and execution in hazardous environments.

Findings

Current VLMs exhibit systematic safety failures.

Existing benchmarks lack multi-stage safety evaluation.

Safety limitations highlight need for improved safety alignment.

Abstract

The integration of vision-language models (VLMs) is driving a new generation of embodied agents capable of operating in human-centered environments. However, as deployment expands, these systems face growing safety risks, particularly when executing hazardous instructions. Current safety evaluation benchmarks remain limited: they cover only narrow scopes of hazards and focus primarily on final outcomes, neglecting the agent's full perception-planning-execution process and thereby obscuring critical failure modes. Therefore, we present SAFE, a benchmark for systematically assessing the safety of embodied VLM agents on hazardous instructions. SAFE comprises three components: SAFE-THOR, an extensible adversarial simulation sandbox with a universal adapter that maps high-level VLM outputs to low-level embodied controls, supporting diverse agent workflow integration; SAFE-VERSE, a risk-aware task suite inspired by Asimov's Three Laws of Robotics, comprising 45 adversarial scenarios, 1,350 hazardous tasks, and 9,900 instructions that span risks to humans, environments, and agents; and SAFE-DIAGNOSE, a multi-level and fine-grained evaluation protocol measuring agent performance across perception, planning, and execution. Applying SAFE to nine state-of-the-art VLMs and two embodied agent workflows, we uncover systematic failures in translating hazard recognition into safe planning and execution. Our findings reveal fundamental limitations in current safety alignment and demonstrate the necessity of a comprehensive, multi-stage evaluation for developing safer embodied intelligence.