IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment

TL;DR

IndoorWorld is a new multi-agent environment that combines physical tasks and social interactions, enabling more realistic AI agent research and applications like architectural design simulation.

Contribution

It introduces a novel heterogeneous environment integrating physical and social dynamics, addressing limitations of existing environments for LLM agent research.

Findings

Multi-agent collaboration impacts agent behavior.

Resource competition influences social dynamics.

Spatial layout affects agent interactions.

Abstract

Virtual environments are essential to AI agent research. Existing environments for LLM agent research typically focus on either physical task solving or social simulation, with the former oversimplifying agent individuality and social dynamics, and the latter lacking physical grounding of social behaviors. We introduce IndoorWorld, a heterogeneous multi-agent environment that tightly integrates physical and social dynamics. By introducing novel challenges for LLM-driven agents in orchestrating social dynamics to influence physical environments and anchoring social interactions within world states, IndoorWorld opens up possibilities of LLM-based building occupant simulation for architectural design. We demonstrate the potential with a series of experiments within an office setting to examine the impact of multi-agent collaboration, resource competition, and spatial layout on agent behavior.