Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models
Cong Lu, Shengran Hu, Jeff Clune

TL;DR
Intelligent Go-Explore (IGE) enhances the original exploration algorithm by integrating giant pretrained foundation models to automatically identify promising states, enabling more efficient and human-like exploration in complex environments.
Contribution
This work introduces IGE, which replaces handcrafted heuristics in Go-Explore with foundation models, significantly broadening its applicability and effectiveness in complex exploration tasks.
Findings
IGE outperforms classical reinforcement learning and graph search baselines.
IGE succeeds in environments where prior foundation model agents fail.
IGE demonstrates strong performance across language and vision-based tasks.
Abstract
Go-Explore is a powerful family of algorithms designed to solve hard-exploration problems built on the principle of archiving discovered states, and iteratively returning to and exploring from the most promising states. This approach has led to superhuman performance across a wide variety of challenging problems including Atari games and robotic control, but requires manually designing heuristics to guide exploration (i.e., determine which states to save and explore from, and what actions to consider next), which is time-consuming and infeasible in general. To resolve this, we propose Intelligent Go-Explore (IGE) which greatly extends the scope of the original Go-Explore by replacing these handcrafted heuristics with the intelligence and internalized human notions of interestingness captured by giant pretrained foundation models (FMs). This provides IGE with a human-like ability to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsComputational Physics and Python Applications
MethodsGo-Explore
