Towards Adaptive Environment Generation for Training Embodied Agents

TL;DR

This paper introduces a feedback-driven, closed-loop environment generation method that adapts training scene difficulty based on the agent's current performance, aiming to improve learning efficiency and generalization.

Contribution

It proposes a novel closed-loop environment generation framework that dynamically adjusts scene difficulty using detailed performance feedback, unlike traditional open-loop methods.

Findings

Enhanced training efficiency through adaptive environment difficulty

Improved agent generalization to new environments

Demonstrated feasibility of feedback-driven environment generation

Abstract

Embodied agents struggle to generalize to new environments, even when those environments share similar underlying structures to their training settings. Most current approaches to generating these training environments follow an open-loop paradigm, without considering the agent's current performance. While procedural generation methods can produce diverse scenes, diversity without feedback from the agent is inefficient. The generated environments may be trivially easy, providing limited learning signal. To address this, we present a proof-of-concept for closed-loop environment generation that adapts difficulty to the agent's current capabilities. Our system employs a controllable environment representation, extracts fine-grained performance feedback beyond binary success or failure, and implements a closed-loop adaptation mechanism that translates this feedback into environment modifications. This feedback-driven approach generates training environments that more challenging in the ways the agent needs to improve, enabling more efficient learning and better generalization to novel settings.