"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents

TL;DR

This paper explores risk awareness and traceability in personalized computer-use agents, proposing a framework to improve user understanding of agent actions, permissions, and residual effects for safer deployment.

Contribution

It introduces AgentTrace, a traceability framework with a prototype interface that enhances understanding and auditability of agent behaviors and their impacts.

Findings

Participants recognized agents as risky but lacked concrete mental models.

Traceability interfaces improved understanding of agent actions.

Enhanced traceability supports anomaly detection and calibrated trust.

Abstract

Personalized computer-use agents are rapidly moving from expert communities into mainstream use. Unlike conventional chatbots, these systems can install skills, invoke tools, access private resources, and modify local environments on users' behalf. Yet users often do not know what authority they have delegated, what the agent actually did during task execution, or whether the system has been safely removed afterward. We investigate this gap as a combined problem of risk understanding and post-hoc auditability, using OpenClaw as a motivating case. We first build a multi-source corpus of the OpenClaw ecosystem, including incidents, advisories, malicious-skill reports, news coverage, tutorials, and social-media narratives. We then conduct an interview study to examine how users and practitioners understand skills, autonomy, privilege, persistence, and uninstallation. Our findings suggest that participants often recognized these systems as risky in the abstract, but lacked concrete mental models of what skills can do, what resources agents can access, and what changes may remain after execution or removal. Motivated by these findings, we propose AgentTrace, a traceability framework and prototype interface for visualizing agent actions, touched resources, permission history, provenance, and persistent side effects. A scenario-based evaluation suggests that traceability-oriented interfaces can improve understanding of agent behavior, support anomaly detection, and foster more calibrated trust.