OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

TL;DR

OS-Sentinel introduces a hybrid safety detection framework for mobile GUI agents that combines formal verification and vision-language models, improving safety assessment accuracy in realistic mobile workflows.

Contribution

The paper presents OS-Sentinel, a novel hybrid safety detection framework that integrates formal verification with VLM-based contextual judgment for mobile agents.

Findings

OS-Sentinel outperforms existing methods by 10%-30% on safety metrics.

The MobileRisk-Live environment enables realistic safety benchmarking.

Hybrid approach enhances detection of both explicit and contextual safety risks.

Abstract

Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents hold great promise for advancing digital automation, their potential for unsafe operations, such as system compromise and privacy leakage, is raising significant concerns. Detecting these safety concerns across the vast and complex operational space of mobile environments presents a formidable challenge that remains critically underexplored. To establish a foundation for mobile agent safety research, we introduce MobileRisk-Live, a dynamic sandbox environment accompanied by a safety detection benchmark comprising realistic trajectories with fine-grained annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety detection framework that synergistically combines a Formal Verifier for detecting explicit system-level violations with a VLM-based Contextual Judge for assessing contextual risks and agent actions. Experiments show that OS-Sentinel achieves 10%-30% improvements over existing approaches across multiple metrics. Further analysis provides critical insights that foster the development of safer and more reliable autonomous mobile agents. Our code and data are available at https://github.com/OS-Copilot/OS-Sentinel.