From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent

TL;DR

This paper introduces PASB, a comprehensive security evaluation framework for personalized AI agents like OpenClaw, revealing critical vulnerabilities through systematic testing in realistic scenarios.

Contribution

It presents PASB, the first end-to-end security benchmarking framework tailored for real-world personalized AI agents, incorporating realistic scenarios and attack evaluations.

Findings

OpenClaw has vulnerabilities in user prompt processing.

Security risks are present in tool usage and memory retrieval stages.

Personalized agent deployments face significant security challenges.

Abstract

Although large language model (LLM)-based agents, exemplified by OpenClaw, are increasingly evolving from task-oriented systems into personalized AI assistants for solving complex real-world tasks, their practical deployment also introduces severe security risks. However, existing agent security research and evaluation frameworks primarily focus on synthetic or task-centric settings, and thus fail to accurately capture the attack surface and risk propagation mechanisms of personalized agents in real-world deployments. To address this gap, we propose Personalized Agent Security Bench (PASB), an end-to-end security evaluation framework tailored for real-world personalized agents. Building upon existing agent attack paradigms, PASB incorporates personalized usage scenarios, realistic toolchains, and long-horizon interactions, enabling black-box, end-to-end security evaluation on real systems. Using OpenClaw as a representative case study, we systematically evaluate its security across multiple personalized scenarios, tool capabilities, and attack types. Our results indicate that OpenClaw exhibits critical vulnerabilities at different execution stages, including user prompt processing, tool usage, and memory retrieval, highlighting substantial security risks in personalized agent deployments. The code for the proposed PASB framework is available at https://github.com/AstorYH/PASB.