Environment Scaling for Interactive Agentic Experience Collection: A Survey

TL;DR

This survey reviews methods for scaling interactive environments to enhance agent learning through the Generation-Execution-Feedback loop, emphasizing realism, complexity, and interactivity for improved agent capabilities.

Contribution

It systematically organizes environment scaling techniques within the GEF loop framework, providing a comprehensive overview and future research directions.

Findings

Environments are crucial for experiential data in agent training.

Scaling environments enhances agent adaptability and decision-making.

The survey identifies key challenges and frameworks in environment scaling.

Abstract

LLM-based agents can autonomously accomplish complex tasks across various domains. However, to further cultivate capabilities such as adaptive behavior and long-term decision-making, training on static datasets built from human-level knowledge is insufficient. These datasets are costly to construct and lack both dynamism and realism. A growing consensus is that agents should instead interact directly with environments and learn from experience through reinforcement learning. We formalize this iterative process as the Generation-Execution-Feedback (GEF) loop, where environments generate tasks to challenge agents, return observations in response to agents' actions during task execution, and provide evaluative feedback on rollouts for subsequent learning. Under this paradigm, environments function as indispensable producers of experiential data, highlighting the need to scale them toward greater complexity, realism, and interactivity. In this survey, we systematically review representative methods for environment scaling from a pioneering environment-centric perspective and organize them along the stages of the GEF loop, namely task generation, task execution, and feedback. We further analyze implementation frameworks, challenges, and applications, consolidating fragmented advances and outlining future research directions for agent intelligence.