RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management

TL;DR

RiskWebWorld is a realistic benchmark designed to evaluate GUI agents in high-stakes e-commerce risk management, highlighting current performance gaps and supporting reinforcement learning improvements.

Contribution

We introduce RiskWebWorld, the first comprehensive interactive benchmark for GUI agents in e-commerce risk management, with a scalable infrastructure and evaluation of diverse models.

Findings

Top-tier models achieve 49.1% success rate.

Specialized open-weights models perform near total failure.

Reinforcement learning improves open-source models by 16.2%.

Abstract

Graphical User Interface (GUI) agents show strong capabilities for automating web tasks, but existing interactive benchmarks primarily target benign, predictable consumer environments. Their effectiveness in high-stakes, investigative domains such as authentic e-commerce risk management remains underexplored. To bridge this gap, we present RiskWebWorld, the first highly realistic interactive benchmark for evaluating GUI agents in e-commerce risk management. RiskWebWorld features 1,513 tasks sourced from production risk-control pipelines across 8 core domains, and captures the authentic challenges of risk operations on uncooperative websites, partially environmental hijackments. To support scalable evaluation and agentic reinforcement learning (RL), we further build a Gymnasium-compliant infrastructure that decouples policy planning from environment mechanics. Our evaluation across diverse models reveals a dramatic capability gap: top-tier generalist models achieve 49.1% success, while specialized open-weights GUI models lag at near-total failure. This highlights that foundation model scale currently matters more than zero-shot interface grounding in long-horizon professional tasks. We also demonstrate the viability of our infrastructure through agentic RL, which improves open-source models by 16.2%. These results position RiskWebWorld as a practical testbed for developing robust digital workers.