LEAF: Latent Exploration Along the Frontier

TL;DR

LEAF introduces a dynamics-aware manifold exploration method that efficiently learns reachable states for goal-conditioned policies from images, improving exploration in robotic tasks without ground-truth state access.

Contribution

The paper proposes a novel exploration framework that learns a reachable state manifold and uses a frontier-based approach for goal-reaching, enhancing policy learning from images without ground-truth states.

Findings

Outperforms existing baselines on robotic environments

Effectively learns a latent reachability manifold

Improves exploration efficiency with curriculum sampling

Abstract

Self-supervised goal proposal and reaching is a key component for exploration and efficient policy learning algorithms. Such a self-supervised approach without access to any oracle goal sampling distribution requires deep exploration and commitment so that long horizon plans can be efficiently discovered. In this paper, we propose an exploration framework, which learns a dynamics-aware manifold of reachable states. For a goal, our proposed method deterministically visits a state at the current frontier of reachable states (commitment/reaching) and then stochastically explores to reach the goal (exploration). This allocates exploration budget near the frontier of the reachable region instead of its interior. We target the challenging problem of policy learning from initial and goal states specified as images, and do not assume any access to the underlying ground-truth states of the robot and the environment. To keep track of reachable latent states, we propose a distance-conditioned reachability network that is trained to infer whether one state is reachable from another within the specified latent space distance. Given an initial state, we obtain a frontier of reachable states from that state. By incorporating a curriculum for sampling easier goals (closer to the start state) before more difficult goals, we demonstrate that the proposed self-supervised exploration algorithm, superior performance compared to existing baselines on a set of challenging robotic environments.https://sites.google.com/view/leaf-exploration