WhatIf: Interactive Exploration of LLM-Powered Social Simulations for Policy Reasoning

TL;DR

WhatIf is an interactive system that allows policymakers to dynamically steer and analyze LLM-powered social simulations in real time, enhancing decision-making in uncertain disaster scenarios.

Contribution

The paper introduces WhatIf, a novel interactive platform for real-time social simulation exploration tailored for policy planning under uncertainty.

Findings

Participants used WhatIf for iterative branching and comparison.

Reflected on tacit planning assumptions and uncovered vulnerabilities.

Grounded reasoning in agent-level cases rather than aggregate outputs.

Abstract

Policymakers in domains such as emergency management, public health, and urban planning must make decisions under deep uncertainty, where outcomes depend on how large populations interpret information, coordinate, and adopt over time. Existing tools only partially support this process: tabletop exercises enable collaborative discussion but lack dynamic feedback, while computational simulations capture population dynamics but are designed for offline analysis. We present WhatIf, an interactive system that enables policymakers to steer, inspect, and compare LLM-powered social simulations in real time. Informed by a formative study in emergency preparedness planning, we derive four design requirements for interactive policy simulations: fluid steering, real-time scale, collaborative exploration, and multi-level interpretability. We developed WhatIf guided by these requirements and evaluated it with five preparedness professionals across three disaster evacuation scenarios. Our findings show that participants used the system as a space for iterative branching and comparison rather than evaluating fixed plans; reflected on tacit planning assumptions when agent behavior violated expectations; surfaced previously unrecognized planning vulnerabilities; and grounded their reasoning in inspectable agent-level cases rather than aggregate outputs alone. These findings suggest broader design implications for LLM-powered social simulation systems: designing such systems as interactive, shared reasoning environments -- rather than offline predictive tools -- can better support expert decision-making under deep uncertainty.