World of Workflows: A Benchmark for Bringing World Models to Enterprise Systems

TL;DR

This paper introduces World of Workflows (WoW), a benchmark environment for testing large language models in complex enterprise systems with hidden workflows, revealing their limitations and emphasizing the need for grounded world modeling.

Contribution

It presents a realistic enterprise benchmark with 4,000+ rules and 55 workflows, highlighting the importance of system dynamics modeling for reliable enterprise agents.

Findings

LLMs fail to predict cascading side effects in enterprise workflows.

Reliability requires agents to simulate hidden state transitions.

WoW provides a new environment for evaluating enterprise system understanding.

Abstract

Frontier large language models (LLMs) excel as autonomous agents in many domains, yet they remain untested in complex enterprise systems where hidden workflows create cascading effects across interconnected databases. Existing enterprise benchmarks evaluate surface-level agentic task completion similar to general consumer benchmarks, ignoring true challenges in enterprises, such as limited observability, large database state, and hidden workflows with cascading side effects. We introduce World of Workflows (WoW), a realistic ServiceNow-based environment incorporating 4,000+ business rules and 55 active workflows embedded in the system, alongside WoW-bench, a benchmark of 234 tasks evaluating constrained agentic task completion and enterprise dynamics modeling capabilities. We reveal two major takeaways: (1) Frontier LLMs suffer from dynamics blindness, consistently failing to predict the invisible, cascading side effects of their actions, which leads to silent constraint violations, and (2) reliability in opaque systems requires grounded world modeling, where agents must mentally simulate hidden state transitions to bridge the observability gap when high-fidelity feedback is unavailable. For reliable and useful enterprise agents, WoW motivates a new paradigm to explicitly learn system dynamics. We release our GitHub for setting up and evaluating WoW.