OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code

TL;DR

OMNI-EPIC introduces a framework that uses foundation models to autonomously generate diverse, interesting, and appropriately challenging environments and tasks in code, advancing open-ended AI development.

Contribution

It presents OMNI-EPIC, a novel system that creates and adapts complex environments and reward functions dynamically, enabling endless, self-driven exploration for AI agents.

Findings

Demonstrates continuous generation of diverse environments and tasks.

Shows adaptation of tasks based on agent learning progress.

Highlights increased creativity in environment design.

Abstract

Open-ended and AI-generating algorithms aim to continuously generate and solve increasingly complex tasks indefinitely, offering a promising path toward more general intelligence. To accomplish this grand vision, learning must occur within a vast array of potential tasks. Existing approaches to automatically generating environments are constrained within manually predefined, often narrow distributions of environment, limiting their ability to create any learning environment. To address this limitation, we introduce a novel framework, OMNI-EPIC, that augments previous work in Open-endedness via Models of human Notions of Interestingness (OMNI) with Environments Programmed in Code (EPIC). OMNI-EPIC leverages foundation models to autonomously generate code specifying the next learnable (i.e., not too easy or difficult for the agent's current skill set) and interesting (e.g., worthwhile and novel) tasks. OMNI-EPIC generates both environments (e.g., an obstacle course) and reward functions (e.g., progress through the obstacle course quickly without touching red objects), enabling it, in principle, to create any simulatable learning task. We showcase the explosive creativity of OMNI-EPIC, which continuously innovates to suggest new, interesting learning challenges. We also highlight how OMNI-EPIC can adapt to reinforcement learning agents' learning progress, generating tasks that are of suitable difficulty. Overall, OMNI-EPIC can endlessly create learnable and interesting environments, further propelling the development of self-improving AI systems and AI-Generating Algorithms. Project website with videos: https://dub.sh/omniepic