When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents

TL;DR

This paper introduces AutoElicit, a framework to systematically identify and analyze severe unintended behaviors in computer-use agents caused by benign inputs, revealing persistent vulnerabilities across state-of-the-art CUAs.

Contribution

It presents the first conceptual and methodological framework for eliciting and analyzing unintended behaviors in CUAs, including an automated perturbation method called AutoElicit.

Findings

AutoElicit surfaces hundreds of harmful behaviors in CUAs.

Persistent vulnerabilities are identified across different state-of-the-art CUAs.

The framework provides a foundation for systematic analysis of unintended behaviors.

Abstract

Although computer-use agents (CUAs) hold significant potential to automate increasingly complex OS workflows, they can demonstrate unsafe unintended behaviors that deviate from expected outcomes even under benign input contexts. However, exploration of this risk remains largely anecdotal, lacking concrete characterization and automated methods to proactively surface long-tail unintended behaviors under realistic CUA scenarios. To fill this gap, we introduce the first conceptual and methodological framework for unintended CUA behaviors, by defining their key characteristics, automatically eliciting them, and analyzing how they arise from benign inputs. We propose AutoElicit: an agentic framework that iteratively perturbs benign instructions using CUA execution feedback, and elicits severe harms while keeping perturbations realistic and benign. Using AutoElicit, we surface hundreds of harmful unintended behaviors from state-of-the-art CUAs such as Claude 4.5 Haiku and Opus. We further evaluate the transferability of human-verified successful perturbations, identifying persistent susceptibility to unintended behaviors across various other frontier CUAs. This work establishes a foundation for systematically analyzing unintended behaviors in realistic computer-use settings.