Heterogeneous Adversarial Play in Interactive Environments

TL;DR

This paper introduces Heterogeneous Adversarial Play, a novel adversarial framework for autonomous curriculum learning that dynamically adapts task difficulty through bidirectional interactions, improving learning efficiency in multi-task environments.

Contribution

It formalizes a new adversarial curriculum learning method with dynamic task adaptation, inspired by human pedagogical asymmetry, and demonstrates its effectiveness in multi-task learning.

Findings

Achieves performance comparable to state-of-the-art baselines.

Generates curricula that improve learning efficacy.

Validates across both artificial agents and human subjects.

Abstract

Self-play constitutes a fundamental paradigm for autonomous skill acquisition, whereby agents iteratively enhance their capabilities through self-directed environmental exploration. Conventional self-play frameworks exploit agent symmetry within zero-sum competitive settings, yet this approach proves inadequate for open-ended learning scenarios characterized by inherent asymmetry. Human pedagogical systems exemplify asymmetric instructional frameworks wherein educators systematically construct challenges calibrated to individual learners' developmental trajectories. The principal challenge resides in operationalizing these asymmetric, adaptive pedagogical mechanisms within artificial systems capable of autonomously synthesizing appropriate curricula without predetermined task hierarchies. Here we present Heterogeneous Adversarial Play (HAP), an adversarial Automatic Curriculum Learning framework that formalizes teacher-student interactions as a minimax optimization wherein task-generating instructor and problem-solving learner co-evolve through adversarial dynamics. In contrast to prevailing ACL methodologies that employ static curricula or unidirectional task selection mechanisms, HAP establishes a bidirectional feedback system wherein instructors continuously recalibrate task complexity in response to real-time learner performance metrics. Experimental validation across multi-task learning domains demonstrates that our framework achieves performance parity with SOTA baselines while generating curricula that enhance learning efficacy in both artificial agents and human subjects.