Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models

TL;DR

The paper introduces IVE, a framework that uses vision-language models to imagine, verify, and execute actions, enabling more diverse exploration and improved policy learning in robotic environments.

Contribution

It presents a novel exploration method that integrates semantic scene understanding with physical plausibility checks to enhance robotic exploration.

Findings

IVE increases state visitation entropy by 4.1 to 7.8 times.

Policies trained on IVE data outperform those trained on human demonstrations.

IVE enables more diverse and meaningful exploration in tabletop environments.

Abstract

Exploration is essential for general-purpose robotic learning, especially in open-ended environments where dense rewards, explicit goals, or task-specific supervision are scarce. Vision-language models (VLMs), with their semantic reasoning over objects, spatial relations, and potential outcomes, present a compelling foundation for generating high-level exploratory behaviors. However, their outputs are often ungrounded, making it difficult to determine whether imagined transitions are physically feasible or informative. To bridge the gap between imagination and execution, we present IVE (Imagine, Verify, Execute), an agentic exploration framework inspired by human curiosity. Human exploration is often driven by the desire to discover novel scene configurations and to deepen understanding of the environment. Similarly, IVE leverages VLMs to abstract RGB-D observations into semantic scene graphs, imagine novel scenes, predict their physical plausibility, and generate executable skill sequences through action tools. We evaluate IVE in both simulated and real-world tabletop environments. The results show that IVE enables more diverse and meaningful exploration than RL baselines, as evidenced by a 4.1 to 7.8x increase in the entropy of visited states. Moreover, the collected experience supports downstream learning, producing policies that closely match or exceed the performance of those trained on human-collected demonstrations.