PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimization

TL;DR

PolicySim is an LLM-based social simulation sandbox designed to proactively evaluate and optimize platform intervention policies, addressing societal risks like echo chambers and polarization before deployment.

Contribution

It introduces a novel LLM-based simulation framework with realistic user and intervention modules for proactive policy assessment.

Findings

Accurately simulates platform ecosystems at multiple levels

Supports effective intervention policy testing

Enhances proactive risk management in social platforms

Abstract

Social platforms serve as central hubs for information exchange, where user behaviors and platform interventions jointly shape opinions. However, intervention policies like recommendation and content filtering, can unintentionally amplify echo chambers and polarization, posing significant societal risks. Proactively evaluating the impact of such policies is therefore crucial. Existing approaches primarily rely on reactive online A/B testing, where risks are identified only after deployment, making risk identification delayed and costly. LLM-based social simulations offer a promising pre-deployment alternative, but current methods fall short in realistically modeling platform interventions and incorporating feedback from the platform. Bridging these gaps is essential for building actionable frameworks to assess and optimize platform policies. To this end, we propose PolicySim, an LLM-based social simulation sandbox for the proactive assessment and optimization of intervention policies. PolicySim models the bidirectional dynamics between user behavior and platform interventions through two key components: (1) a user agent module refined via supervised fine-tuning (SFT) and direct preference optimization (DPO) to achieve platform-specific behavioral realism; and (2) an adaptive intervention module that employs a contextual bandit with message passing to capture dynamic network structures. Experiments show that PolicySim can accurately simulate platform ecosystems at both micro and macro levels and support effective intervention policy.